I served as a Company Grade officer for several years before I got to a unit that actually executed performance counseling and evaluations correctly. For my first several years the OER was an event that happened once, when you signed the cut and pasted evaluation. Almost everyone who was not hated got similar evaluations and there was almost a "lockstep" blocking from the senior rated based on time in grade and who was in the zone of consideration for promotion.
When I reported to the, 507th CSG, 7th Trans. Bn ("On Time, On Target"), I finally was exposed to real performance counseling and evaluations. My BDE commander met with all of the Captains, provided a copy of the Yearly Training Brief, a copy of Initial Counseling (which was filled out after initial counseling, with actual agreed on performance goal- (yes, I know, "agreed on" is a loose term; obviously, the BDE Commander's expectations were the baseline) and we had quarterly meetings to follow up with benchmarks, issues, recommendations for improvement and updated goals documented along the way. The actual OERs issued seemed to accurately reflect both performance and potential at the end of the rating period. (This was, of course, taken up by me and implemented for my junior officers, but also was concurrently executed for NCOER's by the CSM, my 1SG, and the junior NCO's. We actually did performance counseling for everyone E4 and above). This was a real eye opening experience for me and it showed in the excellence of the unit (from BDE on down). Instead of being an "administrative requirement" done once a year, the counseling program actually functioned to develop leaders and was an effective tool. (That experience, and my time in that unit was by far the best experience I ever had in the Army).
(I contrast this experience with what happened after I transferred to the JAG Corps. The counseling program my entire time in the JAG Corps was laughable. I remember being rated against 5 Captains. The SJA asked who wanted the Trial Counsel billet- which is typically the "hottest" job for Company Grade JAGs. I stuck my hand up- my peers all demurred, saying, among other things, that they were not sure they wanted to spend the time, were not sure they wanted to stay in the Army, or had other lame excuses. I got the job, did well....and got a less stellar OER than one of my peers who had declared he wanted to resign. When I asked the SJA what was up with the ratings, he said, well, "I wanted to motivate the other Captain...I know you are going to stay and don't have a promotion board for another year, plus if I give you a great evaluation this year, I won't have any room to give you a better one next year- you will look the same over successive years, and it will help you to show improvement year over year...so I gave him a better rating even though you did better and more work...don't worry, I will take care of you next year." My take away from that was the performance did not matter- ratings were based on BS projections of trying to position people for promotions or trying to entice lower performers to stay in. Instead of a motivating tool, the evaluations became essentially meaningless for a performance measure).
My take away is that evaluations are only as good as the raters who administer them and need to be used correctly, as a development tool, not an "event" that needs to be accomplished.