My advice is to change the filter each time you change the oil. I would change it every 3000 miles. The newer cars today can go 5000. My opinion is when you don't change the filter there is dirty oil etc in that filter and could cause the oil to look dirty like you described. I am not a Technician but a Service Advisor.