Luddites weren't afraid that they would be replaced by machines, they were skilled weavers who had already been replaced by programmable looms. Both their status/role in society and their income were significantly decreased as the available jobs switched from artisanal weaving to operating industrial looms in a factory.