Is that the fault of the experiment? Or was it a weak manipulation and no-result was correct (e.g., changing the purchase button to a slightly different shade of green and expecting a higher conversion rate)?
If you are unable to provide your manager reasons for new development, he's going to find someone else to do your job, someone who will give him a report with charts and numbers that he can use to justify and expand his teams operational and headcount budget.
Giving a manager a report that says "there are little to no modifications that we can make at this time to improve UX" is a CLM for him.
IMO, this is still a failure to understand split testing. It is not to discover if changing a button color matters, but to explore the universe of all possible treatments and how they impact your most important business metrics.
It’s a global optimization problem, not a scientific way to understand how a specific change impacts users. Testers that have this mindset tend to be locally constrained and less likely to have bigger wins.
