You have done all the things right. I don't see any issues in your deployment. You say that sometimes your custom transforms are getting applied to some of the machines but not always.
Which makes me believe that that the Reader 11.0.04 installers are working as expected. I guess, It's the deployment and the distribution of packages which is causing this intermittent issue.
WHat i understand is that, while applying the setup.exe, it's somehow ignoring to apply the custom mst by not reading the setup.ini file
My suggestions,
1. Cleanup your existing package and deployment related to Reader 11.0.04
2. Recreate package and deplyment related to Reader 11.0.04 using some other folder
3. If you are not cleaning up your existing package and deployments. Ensure you have the transform file and setup.ini file in the same folder as the package folder (where it contains setup.exe and other necessary files)
and then update distribution points
4. Make the source directory shared