Up to 4,500 other rank and file soldiers could be dismissed, they say.
A Nigerian military source confirmed the dismissals to the BBC, but would not give an exact figure.
The army was widely criticised when the Islamist group Boko Haram captured vast areas in the country’s north-east last year, despite a military emergency.
The Nigerian army, with military backing from Chad, Cameroon and Niger, has now recaptured most of the area in the north-east which the group had seized, but sporadic attacks and violence have continued.
Many of the dismissals are thought to be connected to the fall of Mubi, the second largest town in Adamawa state, one of three states under the state of emergency.
Boko Haram insurgents captured the town in October after clashes with government forces.
One of the soldiers who has been dismissed, and was present at the fall of Mubi, told the BBC Hausa service that soldiers were simply following orders from their commanders, who had told them to retreat from the town because they lacked adequate weapons to take on the militants.
“We weren’t given an opportunity to defend ourselves. I’ve spent 20 years in the service of the Nigerian army, I’ve never been accused of any offence,” said the soldier, who did not want to be named.