Skip to content

Commit 60857c2

Browse files
committed
updated actions as well as readme
1 parent 5b26291 commit 60857c2

File tree

5 files changed

+731
-346
lines changed

5 files changed

+731
-346
lines changed

.github/workflows/release.yml

Lines changed: 7 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -24,47 +24,31 @@ jobs:
2424
with:
2525
python-version: '3.10'
2626

27-
- name: Set up Go module structure
28-
run: |
29-
# Create proper Go module structure for correct imports
30-
mkdir -p $HOME/go/src/github.com/justrach
31-
cp -r $GITHUB_WORKSPACE $HOME/go/src/github.com/justrach/pathik
32-
echo "GOPATH=$HOME/go" >> $GITHUB_ENV
33-
echo "WORKING_DIR=$HOME/go/src/github.com/justrach/pathik" >> $GITHUB_ENV
34-
3527
- name: Install dependencies
3628
run: |
37-
cd ${{ env.WORKING_DIR }}
3829
python -m pip install --upgrade pip
3930
pip install setuptools wheel twine build requests
4031
4132
- name: Extract version from tag
4233
id: get_version
43-
run: |
44-
cd ${{ env.WORKING_DIR }}
45-
echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_ENV
34+
run: echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_ENV
4635

4736
- name: Build binaries for all platforms
48-
run: |
49-
cd ${{ env.WORKING_DIR }}
50-
python build_binary.py --all
37+
run: python build_binary.py --all
5138

5239
- name: Check output structure
5340
run: |
54-
cd ${{ env.WORKING_DIR }}
5541
find . -name "pathik_bin*" | sort
5642
5743
- name: Debug pathik directory
5844
run: |
59-
cd ${{ env.WORKING_DIR }}
6045
echo "Contents of the bin directory:"
6146
ls -la pathik/bin/ || echo "Directory not found"
6247
echo "Searching for all binary files:"
6348
find pathik -type f -name "pathik_bin*" || echo "No binaries found"
6449
6550
- name: Prepare binaries with unique names
6651
run: |
67-
cd ${{ env.WORKING_DIR }}
6852
mkdir -p release_assets
6953
cp pathik/bin/darwin_amd64/pathik_bin release_assets/pathik_bin_darwin_amd64
7054
cp pathik/bin/darwin_arm64/pathik_bin release_assets/pathik_bin_darwin_arm64
@@ -82,11 +66,10 @@ jobs:
8266
draft: false
8367
prerelease: false
8468
token: ${{ secrets.GITHUB_TOKEN }}
85-
files: ${{ env.WORKING_DIR }}/release_assets/*
69+
files: release_assets/*
8670

8771
- name: Build Python package
8872
run: |
89-
cd ${{ env.WORKING_DIR }}
9073
# Ensure version numbers match
9174
sed -i "s/VERSION = '.*'/VERSION = '${{ env.VERSION }}'/g" setup.py
9275
sed -i "s/__version__ = \".*\"/__version__ = \"${{ env.VERSION }}\"/g" pathik/__init__.py
@@ -99,7 +82,6 @@ jobs:
9982
TWINE_USERNAME: __token__
10083
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
10184
run: |
102-
cd ${{ env.WORKING_DIR }}
10385
twine upload dist/*
10486
10587
publish-npm:
@@ -124,20 +106,11 @@ jobs:
124106
- name: Extract version from tag
125107
run: echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_ENV
126108

127-
- name: Set up working directory
128-
run: |
129-
mkdir -p $HOME/go/src/github.com/justrach
130-
cp -r $GITHUB_WORKSPACE $HOME/go/src/github.com/justrach/pathik
131-
echo "WORKING_DIR=$HOME/go/src/github.com/justrach/pathik" >> $GITHUB_ENV
132-
133109
- name: Create JS package binaries directory
134-
run: |
135-
cd ${{ env.WORKING_DIR }}
136-
mkdir -p pathik-js/bin
110+
run: mkdir -p pathik-js/bin
137111

138112
- name: Download released binaries
139113
run: |
140-
cd ${{ env.WORKING_DIR }}
141114
# Create a temporary directory to download the binaries
142115
mkdir -p temp_binaries
143116
@@ -151,7 +124,6 @@ jobs:
151124

152125
- name: Copy binaries to JS package
153126
run: |
154-
cd ${{ env.WORKING_DIR }}
155127
# Create platform-specific directories
156128
mkdir -p pathik-js/bin/darwin-amd64
157129
mkdir -p pathik-js/bin/darwin-arm64
@@ -177,18 +149,18 @@ jobs:
177149
178150
- name: Update package version
179151
run: |
180-
cd ${{ env.WORKING_DIR }}/pathik-js
152+
cd pathik-js
181153
npm version ${{ env.VERSION }} --no-git-tag-version
182154
cat package.json | grep version
183155
184156
- name: Install dependencies
185157
run: |
186-
cd ${{ env.WORKING_DIR }}/pathik-js
158+
cd pathik-js
187159
npm install
188160
189161
- name: Publish to npm
190162
run: |
191-
cd ${{ env.WORKING_DIR }}/pathik-js
163+
cd pathik-js
192164
npm publish
193165
env:
194166
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

README.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ A high-performance web crawler implemented in Go with Python and JavaScript bind
1515
- Cloudflare R2 integration
1616
- Kafka streaming support
1717
- Memory-efficient (uses ~10x less memory than browser automation tools)
18+
- Automatic binary version management
1819

1920
## Performance Benchmarks
2021

@@ -58,6 +59,31 @@ pip install pathik
5859

5960
The package will automatically download the correct binary for your platform from GitHub releases on first use.
6061

62+
### Binary Version Management
63+
64+
Pathik now automatically handles binary version checking and updates:
65+
66+
- When you install or upgrade the Python package, it will check if the binary matches the package version
67+
- If the versions don't match, it will automatically download the correct binary
68+
- You can manually check and update the binary with:
69+
```python
70+
# Force binary update
71+
import pathik
72+
from pathik.crawler import get_binary_path
73+
binary_path = get_binary_path(force_download=True)
74+
```
75+
76+
- Command line options:
77+
```bash
78+
# Check if binary is up to date
79+
pathik --check-binary
80+
81+
# Force update of the binary
82+
pathik --force-update-binary
83+
```
84+
85+
This ensures you always have the correct binary version with all the latest features, especially when using new functionality like Kafka streaming with session IDs.
86+
6187
## Usage
6288

6389
### Python API

0 commit comments

Comments
 (0)