TogetherLLM¶
TogetherLLM
¶
Bases: OpenAILLM
TogetherLLM LLM implementation running the async API client of OpenAI.
Attributes:
Name | Type | Description |
---|---|---|
model |
the model name to use for the LLM e.g. "mistralai/Mixtral-8x7B-Instruct-v0.1". Supported models can be found here. |
|
base_url |
Optional[RuntimeParameter[str]]
|
the base URL to use for the Together API can be set with |
api_key |
Optional[RuntimeParameter[SecretStr]]
|
the API key to authenticate the requests to the Together API. Defaults to |
_api_key_env_var |
str
|
the name of the environment variable to use for the API key. It is meant to be used internally. |
Examples:
Generate text:
```python
from distilabel.llms import AnyscaleLLM
llm = TogetherLLM(model="mistralai/Mixtral-8x7B-Instruct-v0.1", api_key="api.key")
llm.load()
output = llm.generate(inputs=[[{"role": "user", "content": "Hello world!"}]])
```