A cutting-edge startup based in China, Sand AI, has recently unveiled an innovative video-generating AI model called Magi-1. This model has received accolades from prominent figures in the tech industry like Kai-Fu Lee, the founding director of Microsoft Research Asia. However, JS’s testing revealed that Sand AI may be censoring certain images that could potentially provoke the Chinese authorities from the hosted version of the model.
Magi-1 operates by predicting sequences of frames in an “autoregressive” manner to generate videos. Sand AI claims that this model is capable of producing high-quality, controllable footage that accurately captures physics better than other open-source models available. Despite its advanced capabilities, Magi-1 is quite demanding in terms of hardware requirements. With a size of 24 billion parameters, the model necessitates between four to eight Nvidia H100 GPUs to run effectively. For most users, accessing Magi-1 on Sand AI’s platform is the only feasible option.
Users on Sand AI’s platform are required to provide a “prompt” image to initiate the video generation process. However, not all images are permissible. JS discovered that Sand AI prohibits the upload of images depicting Xi Jinping, Tiananmen Square, Tank Man, the Taiwanese flag, and symbols supporting Hong Kong liberation. This filtering mechanism operates at the image level, as attempts to bypass the restrictions by renaming image files were unsuccessful.
While Sand AI is not the only Chinese startup implementing restrictions on politically sensitive images, their filtering practices are notably stringent. Hailuo AI, another Chinese startup, also restricts the upload of images featuring Xi Jinping on its generative media platform. However, Sand AI’s approach appears to be more aggressive, as they even block images of Tiananmen Square. This level of control is influenced by Chinese regulations that mandate models to adhere to strict information controls. A law enacted in 2023 prohibits models from generating content that undermines the unity of the country and social harmony, aligning with the government’s official narratives.
Interestingly, Chinese models tend to be more lenient in filtering pornographic content compared to their American counterparts. Recent reports have highlighted the lack of basic safeguards in Chinese video generators that prevent the creation of nonconsensual nudity. This contrast in filtering priorities between political and explicit content underscores the complex regulatory environment that Chinese startups navigate in the development of AI models.
In conclusion, Sand AI’s Magi-1 represents a significant advancement in AI technology, but its censorship practices underscore the intricate balance that Chinese startups must strike between innovation and compliance with regulatory requirements. As the AI landscape continues to evolve, it will be crucial for companies like Sand AI to navigate these challenges while upholding ethical standards and regulatory compliance.
