GPT-4 (and GPT 3.5)#
Generator plugin uses OpenAI’s GPT-4 to generate text from a text prompt,
or the continuation of a chat. It can also be used with GPT-3.5 by passing
model configuration parameter.
The plugin will treat each
Block of the input as an element of a chat. If a
Tag of kind “role” and name ( “system” | “user” | “assistant” ), the content will be passed
to OpenAI with the corresponding role. If a
Block does not have a role tag, it will
be passed with the configured default role, which defaults to “user” (see config params below).
The plugin handles retrying for rate limits and uses Steamship’s OpenAI API key by default, eliminating the need for you to have a separate OpenAI account.
The simplest possible example is:
gpt4 = steamship.use_plugin("gpt-4") task = gpt4.generate(text="Tell me a joke") task.wait() joke = task.output.blocks.text
To build a chat interaction, you can persist the prompt components to a
tagging them with their conversational roles:
gpt4 = steamship.use_plugin("gpt-4") chat_file = File.create(client, blocks=[ Block( text="You are an assistant who likes to tell jokes about bananas", tags=[Tag(kind=TagKind.ROLE, name=RoleTag.SYSTEM)] ) ]) chat_file.append_block( text="Do you know any fruit jokes?", tags=[Tag(kind=TagKind.ROLE, name=RoleTag.USER)] ) task = gpt4.generate( input_file_id=chat_file.id, append_output_to_file=True, output_file_id=chat_file.id ) task.wait() joke = task.output.blocks.text
In the example above, in addition to being returned as the result of the
Task, the output
Block is appended to
Blocks will be tagged with the “assistant” role to allow more
content to be easily appended and generated.
The Generator interface supports many other ways to provide input and persist output.
The GPT-4 plugin has a few configuration parameters:
str, An openAI API key to use. If left default, will use Steamship’s API key.
int, default 256, The maximum number of tokens to generate per request. Can be overridden in runtime options.
str, default “gpt-4”, The OpenAI model to use. Can be a pre-existing fine-tuned model.
float, default 0.4, Controls randomness. Lower values produce higher likelihood / more predictable results; higher values produce more variety. Values between 0-1.
int, default 1, Controls the nucleus sampling, where the model considers the results of the tokens with top_p probability mass. Values between 0-1.
int, default 0, Control how likely the model will reuse words. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics. Number between -2.0 and 2.0.
int, default 0, Control how likely the model will reuse words. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim. Number between -2.0 and 2.0.
bool, default True, Pass the generated output back through OpenAI’s moderation endpoint and throw an exception if flagged.
int, default 8, Maximum number of retries to make when generating.
float, default 600, Timeout for requests to OpenAI completion API. Default is 600 seconds.
int, default 1, How many completions to generate for each prompt.
str, default RoleTag.USER, The default role to use for a block that does not have a Tag of kind=’role’
str, default “”, System prompt that will be prepended before every request
Additionally, stopwords can be passed in the
stop parameter in the
options of the
generate call. Other parameters may be overridden on an individual invocation by passing
them in the
options as well.