Manchester United secured a dramatic 1-0 victory over Hull City through substitute Marcus Rashford.
Resilient Hull, led by former United assistant Mike Phelan, looked set to remain unbeaten after winning their first two Premier League games.
However, Rashford came off the bench to score a 92nd-minute winner as he fired home from close range from a Wayne Rooney cross.
Arsenal secured their first win of the season as they beat Watford 3-1 at Vicarage Road.
Arsenal seized the lead on nine minutes when Nordin Amrabat knocked Alexis Sanchez over in the penalty area, with Santi Cazorla firing the resulting spot kick straight down the middle.
They doubled their advantage on 40 minutes when Theo Walcott sent a cross to the far post before Sanchez clipped the ball over Heurelho Gomes and into the net.
Sanchez was involved again as Arsenal extended their advantage on the stroke of half-time, sending in a cross for Mesut Ozil to head home.
Watford improved in the second half and reduced the deficit through debutant Roberto Pereyra on 57 minutes after Etienne Capoue had seen his shot blocked, but Arsene Wenger’s side held firm to claim all three points.
Liverpool paid the price for failing to take their chances as they were held to a 1-1 draw away to Tottenham Hotspur.
Philippe Coutinho had a golden chance to open the scoring on five minutes when he received a Roberto Firmino pass inside the area, but he failed to get enough power on the shot, allowing Michel Vorm to make a fine save with his boot.
Liverpool broke the deadlock through a James Milner penalty just before the break when referee Robert Madley ruled that Erik Lamela had fouled Firmino in the area.
Liverpool continued to attack in the second half, with Joel Matip’s header clipping the top of the bar before Sadio Mane saw an effort ruled out for a marginal offside in the build-up, but Spurs salvaged a point when Danny Rose fired home on 72 minutes.
Chelsea eased to a 3-0 victory over Burnley at Stamford Bridge.
Eden Hazard gave Chelsea the lead on nine minutes, and in some style, with the Belgian collecting a pass from Nemanja Matic inside the Chelsea half before racing through and firing into the bottom corner from outside the area.
Chelsea got a deserved second on 41 minutes when Willian, collecting a pass from Diego Costa on the corner of the area, sent a shot across Tom Heaton and into the far corner.
The hosts rounded off the victory when Victor Moses converted fellow substitute Pedro Rodriguez’s cross.
Leicester City got their season up and running with a 2-1 victory over Swansea City at the King Power Stadium.
Everton beat Stoke City 1-0 through Shay Given’s own goal at Goodison Park.
David Moyes got his first point as Sunderland manager in a 1-1 draw against Southampton at St Mary’s.
Crystal Palace scored a late equaliser to secure a 1-1 draw with Bournemouth at Selhurst Park.