Eight pumpers and tankers attended the fire in the unused two-storey building.
CFA was notified of the fire at around 7:30pm.
Around 25 people were relocated to a different area of a hostel across the road as a precaution and crews will remain at the mill overnight to continue blacking out and monitor the situation.
District 16 Operations Officer, Alfred Mason, said the fire was contained to a factory in one wing of the building.
“When our crews first arrived they entered the building to fight the fire but had to pull out as it was unsafe,” he said.
“The wing of the building was fully engulfed in flames and eventually collapsed.
“That wing was completely destroyed and it took crews over an hour to get the fire under control.
“The crews did a great job containing the fire to one wing and saved the rest of the building.”