This is how I'd do it:
SELECT * FROM #MyTable AS mt CROSS APPLY ( SELECT COUNT(DISTINCT mt2.Col_B) AS dc FROM #MyTable AS mt2 WHERE mt2.Col_A = mt.Col_A -- GROUP BY mt2.Col_A ) AS ca;
GROUP BY clause is redundant given the data provided in the question, but may give you a better execution plan. See the follow-up Q & A CROSS APPLY produces outer join.
Consider voting for OVER clause enhancement request - DISTINCT clause for aggregate functions on the feedback site if you would like thisthat feature added natively to SQL Server.