The problem with your problem (sorry, couldn't resist) is that you haven't properly defined what you expect a month to be. How many months are there between Dec 27 and Feb 28? How about Dec 28 and Feb 28? Dec 29? What about Dec 15 to Jan 13? Dec 15 to Jan 14?
As others have mentioned,
DATEDIFF just measures the number of boundaries that have been passed. The following both return 1:
Even though you and I both know that only one day has passed in the first example, and almost two years have passed in the second.
If you want everything from before September 4th (since today is February 3rd), then:
DECLARE @d DATE = SYSDATETIME;
SELECT <cols> FROM dbo.tablename
WHERE VisitDate < DATEADD(DAY, 1, DATEADD(MONTH, -5, @d));
-- the above evaluates to < '2013-09-04'