Liverpool's Andy Carroll, making his first start in the tournament, scored his second international goal for England on 23 minutes, rising high above Sweden's defence to direct a well-placed header past Andreas Isaksson.
Olof Mellberg, though, appeared to get one back just after half-time, forcing Joe Hart to save from close range and then knocking in the rebound. But an own goal was awarded against Glenn Johnson as he scrambled to keep the ball out.
Olympiakos defender Mellberg then put Erik Hamren's side ahead minutes later, rising unchallenged in the six-yard box after slipping behind the English defence from a deep free-kick.
England desperately tried to get back into the game and John Terry came close, amid frantic defending from the Swedes, until Theo Walcott looped a shot from the edge of the penalty over a crowd and players and wrong-footed Isaksson.
The Arsenal winger then had a hand in England's third, pulling the ball back from the right-hand byline to Manchester United's Danny Welbeck, who cheekily back-heeled it into the net.
France had earlier thought their clash with Ukraine had been abandoned after violent thunder and electric storms lashing Shakhtar Donetsk's Donbass Arena forced Dutch referee Bjorn Kuipers to take both teams off after just five minutes' play.
The two sides only returned nearly an hour later and were level at the half-time interval.
Jeremy Menez, selected ahead of Florent Malouda, put the French 1-0 up on 53 minutes, silencing the home fans and hitting hopes that Oleg Blokhin's side could pull off another victory after their shock 2-0 win against Sweden.
Menez's left-foot drive -- only his second international goal for France -- was followed three minutes later by Yohan Cabaye's first for the national team.
France and England are both on four points in Group D, with Ukraine on three and Sweden yet to get off the mark.
Elsewhere, Croatia were facing a potential inquiry after a UEFA-backed racism monitoring group claimed that up to 500 fans taunted Italy's Mario Bariotelli with racial slurs throughout Thursday's 1-1 draw in Poznan, Poland.
The executive director of Football Against Racism (FARE), Piara Powar, told AFP:
"It was fairly intermittent but progressive throughout the game. It intensified when Balotelli was substituted."
An AFP photographer positioned in front of the majority of Croatia fans during Thursday's match said he heard monkey noises being directed at the Manchester City striker, who is of Ghanaian origin.
At one point, he also saw a steward picking up a banana that had been thrown onto the pitch. Balotelli, 21, has previously said that he would walk off the pitch if he were racially abused by fans.
FARE said UEFA will open an investigation into the claims but there was no immediate confirmation from European football's governing body while the Italian football federation said none of the players, including Balotelli, had complained.
UEFA did confirm a 25,000-euro ($31,400, 20,000-pound) fine for separate crowd trouble during Croatia's opening match against the Republic of Ireland.
Racism and crowd trouble have been key issues during the tournament, which is being co-hosted by Poland and Ukraine, amid fears of racist violence by far-right gangs that are reported to be rife at football grounds in both countries.
"It was a cracking game of football. We were reasonably comfortable at half-time and had a good control of the game before they rocked us on our heels," Hodgson said.
"We knew how dangerous they were from set-pieces and they proved true to that. But it was very good the way the team responded to that.
"It's difficult at this level, we're playing against good teams. Our attacking play showed good intentions and we looked dangerous."
But Sweden counterpart Erik Hamren was disconsolate after watching his side go out.
"I feel really bad for the players," Hamren said. "They played a really good match but went out. The operation went well but the patient died. That's how I feel today," he said.
England's first ever competitive victory over the Swedes means they will qualify for the quarter-finals with only a point in their final group game against co-hosts Ukraine in Donetsk next Tuesday.
However, Hodgson will want to see a marked improvement from his side if they are to secure the draw they need after a performance littered with errors and shaky defending.
Seizing possession in midfield the England skipper looked up to curl in a pinpoint long-range diagonal ball which the twisting Carroll met with a thunderous header that gave Isaksson no chance.
Sweden were forced to look for openings on the break and threatened to score when Ibrahimovic dispossessed the careless Johnson and played in Kim Kallstrom whose shot flew over.
England's positive first half display came unstuck, however, within five minutes of the restart as a rejuvenated Sweden dominated play.
The equaliser came after an Ibrahimovic free-kick bounced back off the wall and was played back to an unmarked Mellberg being played onside by Johnson.
The bearded centre-half's shot beat Hart and Johnson was unable to scramble out of his own net. Mellberg claimed it, but the equaliser was ruled an own-goal by Johnson.
England were rocking and it got worse for Hodgson's men on 59 minutes as Sweden went 2-1 up. Sebastian Larsson swung in a curling free-kick and shambolic defending in the England defence allowed Mellberg to nod home Sweden's second.
Milner, who had capped a poor evening by conceding the free-kick which led to Sweden's second, was then promptly hauled off for Walcott.
The Arsenal winger made an immediate impact and within minutes England were level, Walcott's shot from just outside the area catching Isaksson off-balance to make it 2-2.
Sweden continued to cause England problems on the break and Ibrahimovic forced Hart to dive at full-stretch to turn around a 76th-minute effort.
But two minutes later Walcott rescued England again, darting into the box with an electrifying burst to cross for Welbeck, who flicked in a deft finish with the back of his heel to make it 3-2.