Authorities said 48 people, from the Trenton, Mich., area were on board when the boat apparently hit a wave from a larger vessel.
Trenton, Mich., Mayor Gerald Brown told Detroit television station WJBK that 14 Trenton residents were among those on the boat. Of those, three were killed, six survived, and the conditions of the other five were unknown.
Brown said the group of 14 left Tuesday on a weeklong bus and rail trip to see the changing fall colors along the East Coast. The trip was organized through Trenton's Parks and Recreation Department and arranged through a Canadian company, Brown said.
The weather was clear and the water was calm on Lake George Sunday.
"I saw plenty of life jackets in the water, but nobody was in them," said Rick Sause, whose family runs Cramer's Point Motel in Lake George, near where the accident took place.
The 40-foot cruise boat capsized at around 2:55 p.m., said Warren County Sheriff Larry Cleveland. Law enforcement, EMS and fire squads reported to the scene.
The passengers were senior citizens who were using wheelchairs and walkers, Cleveland said. The captain of the Ethan Allen survived and was picked up by a private vessel.
The bodies of the victims were laid out along the shore. Police blocked off the area to keep away onlookers.
This was the first accident of this magnitude ever to happen in the North Country, authorities said.
The people rescued from the boat were all sent to Glens Falls Hospital.
"Out of the numbers transported to the hospital, about four to six were in what I would call 'need to get to the hospital right away' kind of condition," Cleveland said.
Glens Falls Hospital spokesman Jayson White said that as of 7 p.m. Sunday, 27 victims had been brought to the emergency room. Of those, several required treatment for broken ribs and five would likely be admitted to the hospital at least overnight.
"My daughter and I saw police and ambulance," said Frank Sause, owner of Cramer's Point Motel. "Everyone was just shocked and standing around."
Sause's daughter, Andrea, said she ran down first to see what was going on.
"I saw half of the boat out of the water," she said.
Joanne Rahal was in her boat on Lake George with her family when the tragedy occurred.
"The boat was sideways in the water, and people were screaming," Rahal said. "Bodies were floating by our boat."
She performed CPR on someone, but the person was already in bad shape.
Rahal and some other witnesses reported seeing smoke from the boat, but police couldn't confirm that there was any.
"We had an indication that there was a possible fire, but there was no indication of any injury from fire," Cleveland said. "We will also have an investigation into any criminal matter."
The effort to recover the remains of the boat continued into the night.
Dive teams from the South Glens Fire Department, Lake George Village and Horicon were among those responding.
The Adirondack Saratoga Chapter of the American Red Cross set up an assistance center at the scene at the request of the state Emergency Management Office.
Saratogian reporter Jim Kinney and the Associated Press contributed to this report.