I'm sorry about your fish. I assume that you are talking about white cloud mountain minnows (wcmm's)? When mine were ill, they would droop and lose their color as you describe. That lethargy and color loss can be due to most anything including water quality and bacterial infections. I suggest getting them a larger tank with a filter, aeration, etc. If you can't, at least put in an air stone if there isn't one in there. You should also test the water for pH, hardness, ammonia, and nitrite to see how those levels are. Do you add anything to the water when you do water changes (if you do)? I suggest weekly 50% water changes and the addition of dechlorinator and aquarium salt (a tablespoon per 5 gallons).
My section on why fish might wash out is at
http://www.fishpondinfo.com/health3.htm#washI hope the wcmm's get better!