Skip to content

Commit

Permalink
Add langchain annotation blog post (#342)
Browse files Browse the repository at this point in the history
Closes #340
  • Loading branch information
ljvmiranda921 authored Mar 15, 2023
1 parent b478215 commit b2f54de
Show file tree
Hide file tree
Showing 9 changed files with 323 additions and 0 deletions.
30 changes: 30 additions & 0 deletions _drafts/langchain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
<!--
| Scores | Zero-shot | Supervised | Few-shot |
|----------------|-------------------:|--------------------:|-------------:
| Micro F1-score | $$\mathbf{81.45}$$ | $$79.88$$ | $$61.90$$ |
| Macro F1-score | $$\mathbf{78.74}$$ | $$77.52$$ | $$55.02$$ |
| F1-score (per type) | Zero-shot | Supervised | Few-shot |
|------------------------------------------|--------------------:|-----------:|-----------:
| Supporting argument (`Argument_for`) | $$\mathbf{75.21}$$ | $$73.60$$ | $$48.74$$ |
| No argument (`NoArgument`) | $$\mathbf{86.74}$$ | $$85.66$$ | $$72.50$$ |
| Opposing argument (`Argument_against`) | $$\mathbf{74.26}$$ | $$73.30$$ | $$46.00$$ |
-->

<!--
I have to admit that this one's definitely a **negative result.** I initially
expected that the few-shot predictions will work better because there's added
context from the guidelines. But it's also possible that our prompt (plus our
sequential processing step) became a detriment to get more reliable predictions.
I'm not closing my doors to this hypothesis. There's an interesting distribution
of scores especially across category types. For example, [Levy et al.,
(2018)](#levy2018towards) seems to perform well on `NoArgument` cases. Perhaps
there's something in how the guideline was written that caused this? I might get
back to this again in a more qualitative light.
-->
Binary file added assets/png/langchain/cats_macro_auc.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/png/langchain/cats_macro_f.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/png/langchain/header.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/png/langchain/levy.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/png/langchain/refine.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/png/langchain/setup.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/png/langchain/shnarch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
293 changes: 293 additions & 0 deletions notebook/_posts/2023-04-02-langchain-annotation.md

Large diffs are not rendered by default.

0 comments on commit b2f54de

Please sign in to comment.