In a groundbreaking innovation at the intersection of technology and design, Zhaodi Feng, a graduate of the Royal College of Art, has unveiled Promptac, an interactive toolkit redefining the way users interact with AI image generators. Departing from the conventional reliance on textual prompts, Promptac harnesses the power of hand movements, allowing users to shape and modify AI-generated digital designs with intuitive gestures.
The toolkit comes complete with a unique fingertip controller that has the remarkable ability to recognize real-world colors and materials. In addition, a range of small but potent tools—designed to be twisted, pinched, pressed, and bent—enables users to wield tangible influence over AI-created designs, effectively bridging the gap between the physical and digital realms.
Feng conceived the Promptac toolkit to address a crucial gap between the worlds of digital creation and physical craftsmanship, aspiring to facilitate a more organic and intuitive creative process in tandem with AI technology. While traditional generative AI models typically adhere to textual or visual prompts, Promptac's approach is notably different. Crafted from foam clay and silicone, the tools within the kit effectively act as translators, converting the user's physical actions into prompts that the AI can comprehend.
The innovative potential of Promptac was thoroughly validated by Feng through the application of AI tools like Midjourney and InstructPix2Pix. These tools were employed to generate basic images based on text prompts—objects such as vases, mugs, or plant pots. These images then served as the canvas for Promptac's transformative capabilities.
At the heart of Promptac lies the fingertip controller, a sophisticated device that empowers users to identify real-world colors and materials, which can then be seamlessly applied to AI-generated objects. The controller boasts a color sensor and a radio frequency identification (RFID) sensor, effectively recognizing an array of RFID-tagged materials. These materials generate text prompts that subsequently reshape the appearance of the AI-generated image.
Promptac's toolkit arsenal also features what Feng refers to as "soft sensors," enabling users to convey their intentions via physical gestures that are translated into text prompts. These gestures can manipulate the form of a 3D model representing the AI-generated image. The sensor range includes the twist sensor, invoking spiral shapes; the pinch sensor, engendering irregular shape changes; the press sensor, which flattens forms; and the bend sensor, yielding curved or angled shapes.
Notably, Promptac endeavors to democratize AI image generation, catering to individuals without design backgrounds. Feng aims to streamline the communication of design alterations, minimizing the reliance on precise language and enhancing accessibility. This democratization extends even further, potentially assisting those with communication challenges or disabilities, and reinforcing inclusivity in the sphere of technology.
Promptac's inception stemmed from Feng's exploration of traditional crafts, inspired by the interplay between evolving design tools and the transformative impacts of technology on human creativity. While modern technological advancements have brought forth unparalleled efficiencies, they have simultaneously engendered a detachment from the tangible world, contributing to a perceived erosion of tactile skills.
Feng envisions a future where Promptac evolves beyond its current prototype state, envisioning applications across diverse design domains and larger scales, propelled by collaborations with AI researchers. While the current iteration operates by translating gestures into text prompts that AI models can decipher, Feng is confident in the potential for an AI model capable of directly comprehending physical movements.
In her words, "It's feasible to train a new model that directly translates hand manipulations or other tangible interactions to images." Feng underscores that the impact of AI on design and creativity hinges on its intentional design and usage, emphasizing the necessity for more seamless interaction between humans and AI, allowing technology to amplify the flow of creativity.
Promptac stands as a testament to the ever-evolving relationship between human creativity and technology. As other AI-driven projects like Space10's foldable sofa and innovative ventures that reshape crumpled paper into architectural masterpieces emerge, the fusion of AI and design continues to redefine our creative landscape, reshaping the boundaries of what's achievable.
