If these children, especially your stepson, are at the point of being violent toward other people, it's time to get some help. The school staff should be able to let the mom know that counseling is essential for the children, but if they are not, you may want to ask your attorney about legal steps to take.
Your stepchildren need to continue to see your house as a stable, structured place to be. Keep being consistent with them in enforcing rules and expecting good behavior. Make sure that they see there are rewards for that good behavior in the form of time with you and their dad, taking walks, playing games, or reading bedtime stories together.
Your last question brings up a fine line: Mom's house is hers and she gets to make the rules there, but dad needs to be seen as dad wherever he is and the children need to see that they cannot misbehave in front of him. It would be best if mom and dad could talk without the children around and agree to what rules will be enforced and where.