Microsoft Presents ChatGPT For Images 'Visual ChatGPT'

  • Visual ChatGPT system 
  • System's capabilities 
Microsoft Presents ChatGPT For Images 'Visual ChatGPT'

Microsoft has announced the launch of "Visual ChatGPT," a new system that incorporates several types of Visual Foundation Models (VFMs) with ChatGPT. The used models include Transformers, ControlNet, and Stable Diffusion. This system allows users to interact with ChatGPT beyond language, by enabling them to send and receive text/images through chat, as well as insert visual model prompts into the chat to edit their images.

According to a research paper named "Visual ChatGPT: Talking, Drawing, and Editing with Visual Foundation Models," each visual transformer model has its own unique set of tasks with exact inputs and outputs, similar to ChatGPT, which is only trained on text. Even so, when these models are combined, they offer unbounded capabilities for image generation and modification.

To bridge the breach between ChatGPT and VFMs, the research work proposes utilizing a Prompt Manager with various features. Such as notifying ChatGPT about every VFM's capabilities and detailing the required input-output formats. Besides, converting visual information into language format, and managing the histories, priorities, and conflicts of diverse VFMs.

The Prompt Manager enables ChatGPT to use VFMs effectively and receive responses from them continuously until users' wishes are fulfilled. Users can interact with ChatGPT using images and make complex image-related requests or search visual editing by using a multi-step technique including numerous AI models. Additionally, users can request reactions and corrections on the outcomes.

In summary, Visual ChatGPT system offers a new way for users to interact with ChatGPT, but only time will show whether these features are useful for them. You can also find out about the previous ChatGPT announcement of adding the customization features to the famous AI solution.

Nataliia Huivan
Nataliia Huivan
Professional author in IT Industry

Author of articles and news for Atlasiko Inc. I do my best to create qualified and useful content to help our website visitors to understand more about software development, modern IT tendencies and practices. Constant innovations in the IT field and communication with top specialists inspire me to seek knowledge and share it with others.

Share your thoughts in the comments below!

Have any ideas or suggestions about the article or website? Feel free to write it.

Any Questions?

Get in touch with us by simply filling up the form to start our fruitful cooperation right now.

Please check your email
Get a Free Estimate