The vessel, registered HP-4914, overturned at about 2 pm. Local authorities deployed military, police, border guard and other rescue teams to the scene.
The three survivors were taken to Van Don General Hospital for treatment.
The two remaining passengers, both from Hai Phong, were still missing as rescue teams continued search operations.