While entire India was British colony, Goa, Dadra and Nagar Haveli and Daman and Diu were Portuguese colonies for a very long time. The Portuguese colonized Goa in 1510, while the rest of India was colonized by British in 1757.
The freedom movement in the rest of India against the British was also accompanied by freedom movement of Goans against the Portuguese.
There were 14 revolts against the Portuguese colonialists since the mid-19th century but none of them succeeded because of the British. There were forceful conversion by the Portuguese of Hindus to Christianity and vicious suppression of any agitations for freedom.
After British left in 1947, Nehru made several attempts to bring the Portuguese to table about freeing their holds but it did not work. Finally in 1954 a group of local freedom fighters in Dadra and Nagar Haveli were able to free themselves by overthrowing the Portuguese.
Then in late 1961 Nahru ordered for armed invasion to free Goa of the Portuguese. Operation Vijay was launched which included airstrikes at the Dabolim airport, invasion by the navy from the west followed by ground troops from east. The Portuguese had old fashioned and ill-equipped defense as compared to the Indians. The Portuguese thought of fighting till the end, but they were overpowered by India. Merely after 36 hours after the launch of the attack, the Portuguese Governor Vassalo e Silva accepted defeat and signed a document to handover Goa to India.
The Portuguese rule gave Goa a distinct and unique identity because of which it would not have been feasible to merge the state with Karnataka because the cultural and linguistic differences were far too many. But had the Portuguese rule never came to Goa, it would have remained a union territory itself.