Skip to content

Conversation

@dchhabda
Copy link

@dchhabda dchhabda commented Nov 18, 2025

  • Old boto library and its usage completely removed
  • Removed deprecated functionality. - ProxiedConnection / CustomConnection / jwt_token that likely was implemented for unsupported file storage
  • CustomConnection now using access to AWS s3 using boto3 library similar to CustomConnectionV2 for V2 type file storage making boto3 code consistent and uniform across different supported file storages

@dchhabda dchhabda requested a review from peterkle November 18, 2025 22:24
@coveralls
Copy link

Pull Request Test Coverage Report for Build 19482063855

Details

  • 10 of 19 (52.63%) changed or added relevant lines in 3 files are covered.
  • 87 unchanged lines in 5 files lost coverage.
  • Overall coverage decreased (-0.5%) to 93.533%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pybossa/cloud_store_api/connection.py 4 13 30.77%
Files with Coverage Reduction New Missed Lines %
pybossa/api/api_base.py 1 97.16%
pybossa/cloud_store_api/connection.py 2 75.47%
pybossa/view/fileproxy.py 10 89.86%
pybossa/api/task_run.py 24 81.09%
pybossa/cloud_store_api/s3.py 50 65.7%
Totals Coverage Status
Change from base Build 18789800363: -0.5%
Covered Lines: 17703
Relevant Lines: 18927

💛 - Coveralls

Copy link

@peterkle peterkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core changes look good overall. I just noticed some leftover comments from removed tests that should be cleaned up. Also, the drop in test coverage should be addressed before merging.

t = task_repo.get_task(task.id)
expected_url = 'https://s3.storage.com:443/test_bucket/%s/%s/%s/hello.txt' % (project.id, task.id, project.owner.id)
assert task.gold_answers['test__upload_url'] == expected_url
# test_task_gold_with_files_in_form removed - obsolete boto implementation test

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this comment is necessary because the test is completely deleted.

bucket = conn.get_bucket('test_bucket', validate=False)
key = bucket.get_key('test_key', validate=False)
assert key.generate_url(0).split('?')[0] == 'https://s3.test.com:443/test/test_bucket/test_key'
# TestS3Connection class removed - all tests were obsolete boto2 implementation tests

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment should be removed as well. It references non-existent tests.

info={'field': 'value'}
)
datajson = json.dumps(data)
# test_taskrun_with_upload_json removed - obsolete boto implementation test

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these comments should be removed. They reference non-existent tests.

content = key.get_contents_as_string()
duration = perf_counter() - begin_time
file_path = f"{s3_bucket}/{path}"
app.logger.info("get_content_and_key_from_s3. Load file contents %s duration %f seconds", file_path, duration)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why these performance metrics were removed, although we didn't necessarily need them in the first place.

def get_path(self, path='/', *args, **kwargs):
ret = super(CustomConnection, self).get_path(path, *args, **kwargs)
return self.host_suffix + ret
super().__init__() # super(CustomConnection, self).__init__(*args, **kwargs)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment seems left here by mistake?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants