Skip to content

Commit

Permalink
fix docstring formatting tru_virtual
Browse files Browse the repository at this point in the history
  • Loading branch information
joshreini1 committed Jan 26, 2024
1 parent 28c4dee commit 96c07cd
Showing 1 changed file with 44 additions and 40 deletions.
84 changes: 44 additions & 40 deletions trulens_eval/trulens_eval/tru_virtual.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,23 +91,28 @@ class VirtualRecord(Record):
Utility class for creating `Record`s using selectors. In the example below,
`Select.RecordCalls.retriever` refers to a presumed component of some
virtual model which is assumed to have called the method `get_context`. The
inputs and outputs of that call are specified in as the value with the
selector as key. Other than `calls`, other arguments are the same as for
`Record` but empty values are filled for arguments that are not provided but
are otherwise required.
inputs and outputs of that call are specified as the value with the
selector as the key. Other than `calls`, other arguments are the same as for
`Record`, but empty values are filled for arguments that are not provided
but are otherwise required.
**Usage:**
```python
VirtualRecord(
main_input="Where is Germany?", main_output="Germany is in Europe", calls=
{
Select.RecordCalls.retriever.get_context: dict(
args=["Where is Germany?"], rets=["Germany is a country located
in Europe."]
), Select.RecordCalls.some_other_component.do_something: dict(
args=["Some other inputs."], rets=["Some other output."]
)
}
)
VirtualRecord(
main_input="Where is Germany?",
main_output="Germany is in Europe",
calls={
Select.RecordCalls.retriever.get_context: {
'args': ["Where is Germany?"],
'rets': ["Germany is a country located in Europe."]
},
Select.RecordCalls.some_other_component.do_something: {
'args': ["Some other inputs."],
'rets': ["Some other output."]
}
}
)
```
"""

Expand Down Expand Up @@ -194,52 +199,51 @@ class TruVirtual(App):
executed but for whom previously-computed results can be added using
`add_record`. The `VirtualRecord` class may be useful for creating records
for this. Fields used by non-virtual apps can be specified here, notably:
- `app_id: str` -- Unique identifier for the app.
- `tags: List[str]` -- List of tags.
* app_id: str -- Unique identifier for the app.
* tags: List[str] -- List of tags.
- `metadata: Dict[Any, Any]` -- Open-ended metadata.
* metadata: Dict[Any, Any] -- Open-ended metadata.
- `app_extra_json: JSON` -- Additional json structured information to include in the recorded app structure.
* app_extra_json: JSON -- Additional json structured information to include in the recorded app structure.
- `feedbacks: List[Feedback]` -- Which feedback functions to run when a record is ingested.
* feedbacks: List[Feedback] -- Which feedback functions to run when a record is ingested.
- `feedback_mode: FeedbackMode` -- How to run feedback functions when a record is ingested.
* feedback_mode: FeedbackMode -- How to run feedback functions when a record is ingested.
- `app: JSON` -- See below.
* app: JSON -- See below.
# The `app` field.
You can store any information you would like by passing in a dictionry to
You can store any information you would like by passing in a dictionary to
TruVirtual in the `app` field. This may involve an index of components or
versions, or anything else. You can refer to these values for evaluating
feedback.
You can use `VirtualApp` to create the `app` structure or a plain
dictionary. Using `VirtualApp` lets you use Selectors to define components:
```python
virtual_app = VirtualApp()
virtual_app[Select.RecordCalls.llm.maxtokens] = 1024
virtual_app = VirtualApp()
virtual_app[Select.RecordCalls.llm.maxtokens] = 1024
```
# Example
```python
virtual_app = dict(
llm=dict(
modelname="some llm component model name"
),
template="information about the template I used in my app",
debug="all of these fields are completely optional"
)
virtual_app = dict(
llm=dict(
modelname="some llm component model name"
),
template="information about the template I used in my app",
debug="all of these fields are completely optional"
)
virtual = TruVirtual(
app_id="my_virtual_app",
app=virtual_app
)
virtual = TruVirtual(
app_id="my_virtual_app",
app=virtual_app
)
```
"""

app: VirtualApp = Field(default_factory=VirtualApp)
Expand Down

0 comments on commit 96c07cd

Please sign in to comment.