I'm not sure why but after reducing texture sizes the ms went up instead of down.
I went through all the used textures and looked at their settings. I set the foliage albedo to 512 and their normal map to 256. For the others it was 1024 and 512, depending on what the texture was.
I noticed all the material were already instances of a parent, and those parents had specific parameters, so I didn't go through with trying to have them all have the same parent.