When Sevilia's litter is attacked in the sculptors' square, the dress color of the lady who falls out of the litter is a much brighter blue than the dress actually worn by Sevilia; this is either a stunt woman in the wrong colored dress, or a color correction error in post production.
Cato is shown stabbing himself once in the heart and dying, which is not historically accurate. According to Plutarch, Cato attempted to kill himself by stabbing himself with his own sword, but failed to do so due to an injured hand. One of Cato's slaves found him on the ground and called for a physician to stitch up and bandage Cato's wounds. Cato waited until they left him and then tore off the bandages and the stitches with his fingers and pulled out his own intestines, and bled to death.
Octavia is shown playing with a caged cockatiel. These birds are native to Australia. It is unlikely that the Romans had any sort trade directly or through intermediaries with that continent.
The elephant shown in the aftermath of the Battle of Thapsus in North Africa is an Indian elephant. However, during antiquity Indian elephants were traded to the Mediterranean area and were used together with local North African elephants (now extinct). Indian elephants were welcomed because North African elephants were smaller and scarce.
Though the series is known for its historical accuracy, trough it several weapons are shown out of time, such as Roman shields from later periods, daggers from previous periods, Roman swords with modern fittings, and even at some point a Chinese sword from the 19th century.
The street gang marching towards Vorenus's house sounds like they are marching in lockstep. They are clearly not.