Winston Peters has refused to apologise for his attack on a Chinese-Kiwi MP despite the prime minister and others describing his comments as "racist".