Some people like to simplify the dynamic as one group is racist and the other isn't but it's not that simple.
The Thalmor are the actual evil group that orchestrated the civil war in the first place, as well as the ones who created the racism (through multiple means) in the Stormcloaks to encourage the conflict (while the Thalmor are elven supremacists themselves).
The Thalmor only benefit as long as the civil war continues, it ending is to their detriment regardless of who wins.
The Imperials are knowingly suppressing the rights of and hurting the people of Skyrim at the Thalmor's behest because they believe their sacrifice to buy them some time is an acceptable loss to their long term goal of defeating the Thalmor, but the Stormcloaks don't know this, nor have the Imperials made an attempt to communicate this to them (regardless of if they would agree to that).
But as a result of that behavior the Stormcloaks have rejected the idea of continuing to be ruled by the Empire or supporting a government that clearly does not care about them.