You can use the StableDiffusionTool (opens in a new tab) to generate and return Stable Diffusion images from your agent.

Building a ReACT Agent with StableDiffusionTool

Just provide the StableDiffusionTool (opens in a new tab) to your FunctionsBasedAgent (opens in a new tab) to use it.

from steamship.agents.tools.image_generation import StableDiffusionTool
 
self._agent = FunctionsBasedAgent(
    tools=[
        StableDiffusionTool(),
    ],
    llm=OpenAI(self.client),
)

By default, this will cause the ReACT agent to generate an image when you request a picture of something. It will do so using Steamship's managed Stable Diffusion plugin.

If you want to change or constrain the circumstances when this tool is used, you can set the ai_description field during initialization:

DalleTool(
    ai_description=(
        "Used to generate company logos. "
        "Only use if the user has asked for a company logo. "
        "The input is a description of what the logo should look like, in detail. "
        "The output is the desired logo."
    )
)

Full-working examples of DalleTool

The following examples can be copy-pasted into your project:

• CaptainPicardWithVoice (opens in a new tab) - Uses Stable Diffusion to generate pictures when you ask him what he sees outside the space ship.

Deploying an AgentService with StableDiffusionTool

To deploy an agent with this tool, just make sure it's properly wrapped in an AgentService wrapper and then follow the normal deployment instructions. You will get a URL that you can visit to create instances of your agent in the cloud and connect to it from the web & chat apps.