Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,5 +90,3 @@ jobs:
run: |
cd R/tests
Rscript run_tests.R


25 changes: 25 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,33 @@ repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-added-large-files
- id: check-ast
- id: check-case-conflict
- id: check-illegal-windows-names
- id: check-json
- id: check-merge-conflict
- id: check-symlinks
- id: check-toml
- id: check-xml
- id: check-yaml
- id: debug-statements
- id: detect-private-key
- id: end-of-file-fixer
- id: fix-byte-order-marker
- id: forbid-submodules
- id: mixed-line-ending
- id: requirements-txt-fixer
# - id: check-builtin-literals
# - id: check-executables-have-shebangs
# - id: check-shebang-scripts-are-executable
# - id: check-vcs-permalinks
# - id: detect-aws-credentials
# - id: name-tests-test
# - id: pretty-format-json
# - id: trailing-whitespace
# exclude: metaflow/plugins/cards/card_modules/main.js

- repo: https://github.com/ambv/black
rev: 25.12.0
hooks:
Expand Down
2 changes: 1 addition & 1 deletion ADOPTERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,4 +69,4 @@ Below is a partial list of organizations using Metaflow in production. If you'd
- [Zendesk](https://www.zendesk.com)
- [Zillow](https://www.zillow.com)
- [Zipline](https://www.zipline.com)
- [Zynga](https://www.zynga.com)
- [Zynga](https://www.zynga.com)
2 changes: 1 addition & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ include metaflow/plugins/cards/card_modules/*.html
include metaflow/plugins/cards/card_modules/*.js
include metaflow/plugins/cards/card_modules/*.css
include metaflow/plugins/cards/card_viewer/*.html
recursive-include devtools *
recursive-include devtools *
2 changes: 1 addition & 1 deletion R/LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -198,4 +198,4 @@
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
limitations under the License.
2 changes: 1 addition & 1 deletion R/R/decorators-environment.R
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,4 @@ environment_variables <- function(...) {
}

decorator("environment", vars = env_var_dict, .convert_args = FALSE)
}
}
1 change: 0 additions & 1 deletion R/R/decorators-errors.R
Original file line number Diff line number Diff line change
Expand Up @@ -99,4 +99,3 @@ retry <- function(times = 3L, minutes_between_retries = 2L) {
catch <- function(var = NULL, print_exception = TRUE) {
decorator("catch", var = var, print_exception = print_exception)
}

2 changes: 1 addition & 1 deletion R/R/imports.R
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@
#' @export
#' @importFrom magrittr %>%
#' @usage lhs \%>\% rhs
NULL
NULL
1 change: 0 additions & 1 deletion R/doc/metaflow.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,3 @@ knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)

2 changes: 1 addition & 1 deletion R/inst/tutorials/00-helloworld/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ printing out 'Metaflow says: Hi!' to the terminal.**
2. ```Rscript helloworld.R show```
3. ```Rscript helloworld.R run```

If you are using RStudio, you can run this script by directly executing `source("helloworld.R")`.
If you are using RStudio, you can run this script by directly executing `source("helloworld.R")`.
2 changes: 1 addition & 1 deletion R/inst/tutorials/01-playlist/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ Inside a terminal:
If you are using RStudio, you can replace the `run()` in last line in `playlist.R` with `run(genre="comedy")`, and run the episode by executing `source("playlist.R")` in RStudio.

##### Inspect the results
Open the R Markdown file ```playlist.Rmd``` in RStudio and execute the markdown cells.
Open the R Markdown file ```playlist.Rmd``` in RStudio and execute the markdown cells.
1 change: 0 additions & 1 deletion R/inst/tutorials/01-playlist/playlist.R
Original file line number Diff line number Diff line change
Expand Up @@ -76,4 +76,3 @@ metaflow("PlayListFlow") %>%
step(step = "end",
r_function = end) %>%
run()

1 change: 0 additions & 1 deletion R/inst/tutorials/01-playlist/playlist.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,3 @@ message("Playlist for movies in genre: ", run$artifact("genre"))
playlist <- run$artifact("playlist")
print(head(playlist))
```

2 changes: 1 addition & 1 deletion R/inst/tutorials/02-statistics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,4 @@ In a terminal:
If you are using RStudio, you can run this script by directly executing `source("stats.R")`.

##### Inspect the results:
Open the R Markdown file ```stats.Rmd``` in RStudio and execute the markdown cells.
Open the R Markdown file ```stats.Rmd``` in RStudio and execute the markdown cells.
1 change: 0 additions & 1 deletion R/inst/tutorials/02-statistics/stats.R
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,3 @@ metaflow("MovieStatsFlow") %>%
step(step = "end") %>%
run()


2 changes: 1 addition & 1 deletion R/inst/tutorials/03-playlist-redux/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ In a terminal:

If you are using RStudio, you can run this script by directly executing `source("playlist.R")`.

In this ```PlayListReduxFlow```, we reuse the genre median gross box office statistics computed from ```MoviesStatsFlow```, pick the genre with the highest median gross box office, and create a randomized playlist of movies of this picked genre.
In this ```PlayListReduxFlow```, we reuse the genre median gross box office statistics computed from ```MoviesStatsFlow```, pick the genre with the highest median gross box office, and create a randomized playlist of movies of this picked genre.
2 changes: 1 addition & 1 deletion R/inst/tutorials/03-playlist-redux/playlist.R
Original file line number Diff line number Diff line change
Expand Up @@ -55,4 +55,4 @@ metaflow("PlayListReduxFlow") %>%
step(step = "end",
r_function = end) %>%
run()


2 changes: 1 addition & 1 deletion R/inst/tutorials/04-helloaws/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ In a terminal:
If you are using RStudio, you can run this script by directly executing `source("helloaws.R")`.

##### Inspect the results:
Open the R Markdown file ```helloaws.Rmd``` in RStudio and execute the markdown cells.
Open the R Markdown file ```helloaws.Rmd``` in RStudio and execute the markdown cells.
2 changes: 1 addition & 1 deletion R/inst/tutorials/05-statistics-redux/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ and run by `source("stats.R")`.

##### Inspect the results:
Open the R markdown file ```02-statistics/stats.Rmd``` in your RStudio and re-run the cells. You can access
the artifacts stored in AWS S3 from your local RStudio session.
the artifacts stored in AWS S3 from your local RStudio session.
2 changes: 1 addition & 1 deletion R/inst/tutorials/06-worldview/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ monitor all of your Metaflow flows.**

#### To play this episode:
1. ```cd tutorials/06-worldview/```
2. Open ```worldview.Rmd``` in RStudio on your local computer
2. Open ```worldview.Rmd``` in RStudio on your local computer
2 changes: 1 addition & 1 deletion R/inst/tutorials/06-worldview/worldview.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,4 @@ for (run_id in flow$runs){
message("Tags: ")
print(run$tags)
}
```
```
2 changes: 1 addition & 1 deletion R/inst/tutorials/07-autopilot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,4 @@ for SFN trigger. You can then directly run `source("stats.R`)` in RStudio.

##### Inspect the results:
Open the R Markdown file```07-autopilot/stats.Rmd``` in your RStudio and re-run the cells. You can access
the artifacts stored in AWS S3 from your local RStudio session.
the artifacts stored in AWS S3 from your local RStudio session.
2 changes: 1 addition & 1 deletion R/inst/tutorials/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ metaflow::pull_tutorials()
```
This creates a directory tutorials in your current working directory with a subdirectory for each tutorial.

Each tutorial has a brief description and instructions included in the `README.md` in each subfolder.
Each tutorial has a brief description and instructions included in the `README.md` in each subfolder.
1 change: 0 additions & 1 deletion R/man/cash-.metaflow.flowspec.FlowSpec.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion R/man/cash-set-.metaflow.flowspec.FlowSpec.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion R/man/gather_inputs.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion R/man/metaflow.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion R/man/parameter.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion R/man/run.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion R/man/step.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion R/man/sub-sub-.metaflow.flowspec.FlowSpec.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion R/man/sub-subset-.metaflow.flowspec.FlowSpec.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion R/tests/contexts.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,4 @@
]
}
]
}
}
2 changes: 1 addition & 1 deletion R/tests/run_tests.R
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ virtualenv_install("r-metaflow", c("../..", "pandas", "numpy"))
use_virtualenv("r-metaflow")

source("testthat.R")
source("run_integration_tests.R")
source("run_integration_tests.R")
2 changes: 1 addition & 1 deletion R/tests/testthat/test-decorators-environment.R
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ test_that("@environment wrapper parses correctly", {
actual <- environment_variables(foo = "TRUE")[1]
expected <- "@environment(vars={'foo': 'TRUE'})"
expect_equal(actual, expected)
})
})
2 changes: 1 addition & 1 deletion devtools/pick_services.sh
Original file line number Diff line number Diff line change
Expand Up @@ -102,4 +102,4 @@ fi
PRINTABLE="$(pretty_print "${SELECTED_SERVICES[@]}")"
gum style "✅ Deploying $PRINTABLE" --foreground "$COLOR" >&2

echo "$(IFS=,; echo "${SELECTED_SERVICES[*]}")"
echo "$(IFS=,; echo "${SELECTED_SERVICES[*]}")"
1 change: 0 additions & 1 deletion docs/concurrency.md
Original file line number Diff line number Diff line change
Expand Up @@ -327,4 +327,3 @@ Async no yes yes no
partially isolated from the parent which makes `parallel_map` a bad
candidate for execution of arbitrary code.
7. Values returned by `parallel_map` need to be picklable.

2 changes: 1 addition & 1 deletion docs/lifecycle.dot
Original file line number Diff line number Diff line change
Expand Up @@ -205,4 +205,4 @@ digraph Metaflow {
stop_task_heartbeat -> register_artifacts
register_artifacts -> task_finished

}
}
2 changes: 1 addition & 1 deletion docs/sidecars.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,4 +43,4 @@ request failing. A sidecar that handles communication with the metadata
service was a perfect solution.




2 changes: 1 addition & 1 deletion docs/update_lifecycle_png
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# install graphviz first
dot -Tpng lifecycle.dot -o lifecycle.png
dot -Tpng lifecycle.dot -o lifecycle.png
2 changes: 1 addition & 1 deletion metaflow/_vendor/pip.LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
2 changes: 1 addition & 1 deletion metaflow/_vendor/v3_6/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
# Empty file
# Empty file
2 changes: 1 addition & 1 deletion metaflow/_vendor/v3_7/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
# Empty file
# Empty file
1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -424,4 +424,3 @@ def to_yaml(cls, dumper, data):
"""
return dumper.represent_yaml_object(cls.yaml_tag, data, cls,
flow_style=cls.yaml_flow_style)

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/composer.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,4 +136,3 @@ def compose_mapping_node(self, anchor):
end_event = self.get_event()
node.end_mark = end_event.end_mark
return node

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/cyaml.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,4 +98,3 @@ def __init__(self, stream,
Representer.__init__(self, default_style=default_style,
default_flow_style=default_flow_style, sort_keys=sort_keys)
Resolver.__init__(self)

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/dumper.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,4 +59,3 @@ def __init__(self, stream,
Representer.__init__(self, default_style=default_style,
default_flow_style=default_flow_style, sort_keys=sort_keys)
Resolver.__init__(self)

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/error.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,4 +72,3 @@ def __str__(self):
if self.note is not None:
lines.append(self.note)
return '\n'.join(lines)

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/events.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,4 +83,3 @@ class MappingStartEvent(CollectionStartEvent):

class MappingEndEvent(CollectionEndEvent):
pass

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/nodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,3 @@ class SequenceNode(CollectionNode):

class MappingNode(CollectionNode):
id = 'mapping'

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -586,4 +586,3 @@ def parse_flow_mapping_empty_value(self):

def process_empty_scalar(self, mark):
return ScalarEvent(None, None, (True, False), '', mark, mark)

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/representer.py
Original file line number Diff line number Diff line change
Expand Up @@ -386,4 +386,3 @@ def represent_ordered_dict(self, data):

Representer.add_multi_representer(object,
Representer.represent_object)

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/resolver.py
Original file line number Diff line number Diff line change
Expand Up @@ -224,4 +224,3 @@ class Resolver(BaseResolver):
'tag:yaml.org,2002:yaml',
re.compile(r'^(?:!|&|\*)$'),
list('!&*'))

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/serializer.py
Original file line number Diff line number Diff line change
Expand Up @@ -108,4 +108,3 @@ def serialize_node(self, node, parent, index):
self.serialize_node(value, node, key)
self.emit(MappingEndEvent())
self.ascend_resolver()

1 change: 0 additions & 1 deletion metaflow/_vendor/yaml/tokens.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,4 +101,3 @@ def __init__(self, value, plain, start_mark, end_mark, style=None):
self.start_mark = start_mark
self.end_mark = end_mark
self.style = style

1 change: 0 additions & 1 deletion metaflow/cmd/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +0,0 @@

2 changes: 1 addition & 1 deletion metaflow/plugins/cards/ui/.eslintignore
Original file line number Diff line number Diff line change
@@ -1 +1 @@
rollup.config.js
rollup.config.js
Loading
Loading