Verify project setup instructions are still valid

[janbrasna]

I'm not sure springfield can be set up only following the instructions as they are listed now, as both docker and virtualenv scenarios don't work for me starting from scratch. It's either missing some DB prerequisites, or a formerly-optional service (like the queue worker) is no longer optional.

(I'll try to follow the steps in a clean profile to log the issues in detail.)

[janbrasna]

And for bedrock trying shelling into a container e.g. from https://mozmeao.github.io/platform-docs/cms/images/#ive-downloaded-a-fresh-db-and-the-images-are-missing the ./manage.py download_media_to_local errors out for me:

so it may need more things installed. I just don't see where that's covered between the basic setup steps, and running some of these commands. I just may be missing the right section altogether that I'd be supposed to follow. — Or this management command is not supposed to work in the containers pulled, do they miss some service perhaps?

[janbrasna]

I think since psycopg-binary is no longer in the requirements:

• Running make preflight fails with pyscopg dependency error springfield#99 (comment)
• (fix) Springfield: try to deal with missing C support for psycopg springfield#104

there is some extra installation necessary to have pg_config available for the build to succeed, in case there's no postrgres install available on the system.

Not sure if just separate apt-install libpq-dev is enough on Linux, Mac needs either brew install postgresql or if not, perhaps separately brew install libpq && brew link --force libpq to be able to install the prod (thus dev) requirements. But basically a preexisting Postgres is expected now, without specifying / documenting it in the installation steps.

[janbrasna]

Trying docker images instead, I however get (springfield only, since circa early October) no matter what I try:

app-1     | Exception in thread django-main-thread:
app-1     | Traceback (most recent call last):
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 105, in _execute
app-1     |     return self.cursor.execute(sql, params)
app-1     |            ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/backends/sqlite3/base.py", line 360, in execute
app-1     |     return super().execute(query, params)
app-1     |            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
app-1     | sqlite3.OperationalError: no such table: product_details_productdetailsfile
app-1     | 
app-1     | The above exception was the direct cause of the following exception:
app-1     | 
app-1     | Traceback (most recent call last):
app-1     |   File "/usr/local/lib/python3.13/threading.py", line 1044, in _bootstrap_inner
app-1     |     self.run()
app-1     |     ~~~~~~~~^^
app-1     |   File "/usr/local/lib/python3.13/threading.py", line 995, in run
app-1     |     self._target(*self._args, **self._kwargs)
app-1     |     ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/autoreload.py", line 64, in wrapper
app-1     |     fn(*args, **kwargs)
app-1     |     ~~^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/core/management/commands/runserver.py", line 124, in inner_run
app-1     |     autoreload.raise_last_exception()
app-1     |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/autoreload.py", line 86, in raise_last_exception
app-1     |     raise _exception[1]
app-1     |   File "/venv/lib/python3.13/site-packages/django/core/management/__init__.py", line 394, in execute
app-1     |     autoreload.check_errors(django.setup)()
app-1     |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/autoreload.py", line 64, in wrapper
app-1     |     fn(*args, **kwargs)
app-1     |     ~~^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/__init__.py", line 24, in setup
app-1     |     apps.populate(settings.INSTALLED_APPS)
app-1     |     ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/apps/registry.py", line 116, in populate
app-1     |     app_config.import_models()
app-1     |     ~~~~~~~~~~~~~~~~~~~~~~~~^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/apps/config.py", line 269, in import_models
app-1     |     self.models_module = import_module(models_module_name)
app-1     |                          ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/usr/local/lib/python3.13/importlib/__init__.py", line 88, in import_module
app-1     |     return _bootstrap._gcd_import(name[level:], package, level)
app-1     |            ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
app-1     |   File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
app-1     |   File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
app-1     |   File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
app-1     |   File "<frozen importlib._bootstrap_external>", line 1023, in exec_module
app-1     |   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
app-1     |   File "/venv/lib/python3.13/site-packages/wagtail_localize_smartling/models.py", line 146, in <module>
app-1     |     required=smartling_settings.REQUIRED,
app-1     |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/functional.py", line 251, in inner
app-1     |     self._setup()
app-1     |     ~~~~~~~~~~~^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/functional.py", line 404, in _setup
app-1     |     self._wrapped = self._setupfunc()
app-1     |                     ~~~~~~~~~~~~~~~^^
app-1     |   File "/venv/lib/python3.13/site-packages/wagtail_localize_smartling/settings.py", line 134, in _init_settings
app-1     |     for locale_id, _locale in getattr(
app-1     |                               ~~~~~~~^
app-1     |         django_settings, "WAGTAIL_CONTENT_LANGUAGES", []
app-1     |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |     ):
app-1     |     ^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/functional.py", line 187, in __wrapper__
app-1     |     result = func(*self._args, **self._kw)
app-1     |   File "/app/springfield/settings/base.py", line 1309, in lazy_wagtail_langs
app-1     |     enabled_language_codes = [x[0] for x in LANGUAGES]
app-1     |                                             ^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/functional.py", line 187, in __wrapper__
app-1     |     result = func(*self._args, **self._kw)
app-1     |   File "/app/springfield/settings/base.py", line 424, in lazy_langs
app-1     |     return [(lang, product_details.languages[lang]["native"]) for lang in langs if lang in product_details.languages]
app-1     |                                                                                            ^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/product_details/__init__.py", line 50, in __getattr__
app-1     |     data = self._storage.data("{0}.json".format(key))
app-1     |   File "/venv/lib/python3.13/site-packages/product_details/storage.py", line 83, in data
app-1     |     data = self.dir_data(dirname)
app-1     |   File "/venv/lib/python3.13/site-packages/product_details/storage.py", line 131, in dir_data
app-1     |     for fo in qs:
app-1     |               ^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/models/query.py", line 386, in __iter__
app-1     |     self._fetch_all()
app-1     |     ~~~~~~~~~~~~~~~^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/models/query.py", line 1954, in _fetch_all
app-1     |     self._result_cache = list(self._iterable_class(self))
app-1     |                          ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/models/query.py", line 93, in __iter__
app-1     |     results = compiler.execute_sql(
app-1     |         chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size
app-1     |     )
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py", line 1623, in execute_sql
app-1     |     cursor.execute(sql, params)
app-1     |     ~~~~~~~~~~~~~~^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 122, in execute
app-1     |     return super().execute(sql, params)
app-1     |            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 79, in execute
app-1     |     return self._execute_with_wrappers(
app-1     |            ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
app-1     |         sql, params, many=False, executor=self._execute
app-1     |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |     )
app-1     |     ^
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
app-1     |     return executor(sql, params, many, context)
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 100, in _execute
app-1     |     with self.db.wrap_database_errors:
app-1     |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/utils.py", line 91, in __exit__
app-1     |     raise dj_exc_value.with_traceback(traceback) from exc_value
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/backends/utils.py", line 105, in _execute
app-1     |     return self.cursor.execute(sql, params)
app-1     |            ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/backends/sqlite3/base.py", line 360, in execute
app-1     |     return super().execute(query, params)
app-1     |            ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
app-1     | django.db.utils.OperationalError: no such table: product_details_productdetailsfile
app-1     | Traceback (most recent call last):
app-1     |   File "/venv/lib/python3.13/site-packages/django/template/utils.py", line 69, in __getitem__
app-1     |     return self._engines[alias]
app-1     |            ~~~~~~~~~~~~~^^^^^^^
app-1     | KeyError: 'jinja2'
app-1     | 
app-1     | During handling of the above exception, another exception occurred:
app-1     | 
app-1     | Traceback (most recent call last):
app-1     |   File "/app/manage.py", line 14, in <module>
app-1     |     execute_from_command_line(sys.argv)
app-1     |     ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
app-1     |     utility.execute()
app-1     |     ~~~~~~~~~~~~~~~^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/core/management/__init__.py", line 436, in execute
app-1     |     self.fetch_command(subcommand).run_from_argv(self.argv)
app-1     |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/core/management/base.py", line 416, in run_from_argv
app-1     |     self.execute(*args, **cmd_options)
app-1     |     ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/core/management/commands/runserver.py", line 69, in execute
app-1     |     super().execute(*args, **options)
app-1     |     ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/core/management/base.py", line 460, in execute
app-1     |     output = self.handle(*args, **options)
app-1     |   File "/venv/lib/python3.13/site-packages/django/core/management/commands/runserver.py", line 110, in handle
app-1     |     self.run(**options)
app-1     |     ~~~~~~~~^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/core/management/commands/runserver.py", line 117, in run
app-1     |     autoreload.run_with_reloader(self.inner_run, **options)
app-1     |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/autoreload.py", line 670, in run_with_reloader
app-1     |     start_django(reloader, main_func, *args, **kwargs)
app-1     |     ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/autoreload.py", line 659, in start_django
app-1     |     reloader.run(django_main_thread)
app-1     |     ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/autoreload.py", line 342, in run
app-1     |     autoreload_started.send(sender=self)
app-1     |     ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/dispatch/dispatcher.py", line 189, in send
app-1     |     response = receiver(signal=self, sender=sender, **named)
app-1     |   File "/venv/lib/python3.13/site-packages/django/template/autoreload.py", line 50, in watch_for_template_changes
app-1     |     for directory in get_template_directories():
app-1     |                      ~~~~~~~~~~~~~~~~~~~~~~~~^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/template/autoreload.py", line 16, in get_template_directories
app-1     |     for backend in engines.all():
app-1     |                    ~~~~~~~~~~~^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/template/utils.py", line 94, in all
app-1     |     return [self[alias] for alias in self]
app-1     |             ~~~~^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/django/template/utils.py", line 85, in __getitem__
app-1     |     engine = engine_cls(params)
app-1     |   File "/venv/lib/python3.13/site-packages/django_jinja/backend.py", line 174, in __init__
app-1     |     environment_cls = import_string(environment_clspath)
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/module_loading.py", line 30, in import_string
app-1     |     return cached_import(module_path, class_name)
app-1     |   File "/venv/lib/python3.13/site-packages/django/utils/module_loading.py", line 15, in cached_import
app-1     |     module = import_module(module_path)
app-1     |   File "/usr/local/lib/python3.13/importlib/__init__.py", line 88, in import_module
app-1     |     return _bootstrap._gcd_import(name[level:], package, level)
app-1     |            ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
app-1     |   File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
app-1     |   File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
app-1     |   File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
app-1     |   File "<frozen importlib._bootstrap_external>", line 1023, in exec_module
app-1     |   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
app-1     |   File "/app/springfield/jinja2.py", line 8, in <module>
app-1     |     from springfield.cms.templatetags.cms_tags import richtext
app-1     |   File "/app/springfield/cms/templatetags/cms_tags.py", line 19, in <module>
app-1     |     from springfield.cms.models.pages import BASE_UTM_PARAMETERS
app-1     |   File "/app/springfield/cms/models/__init__.py", line 5, in <module>
app-1     |     from .snippets import *  # noqa
app-1     |     ^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/app/springfield/cms/models/snippets.py", line 14, in <module>
app-1     |     from springfield.cms.blocks import HEADING_TEXT_FEATURES
app-1     |   File "/app/springfield/cms/blocks.py", line 11, in <module>
app-1     |     from wagtail_link_block.blocks import LinkBlock
app-1     |   File "/venv/lib/python3.13/site-packages/wagtail_link_block/blocks.py", line 11, in <module>
app-1     |     from wagtail.documents.blocks import DocumentChooserBlock
app-1     |   File "/venv/lib/python3.13/site-packages/wagtail/documents/blocks.py", line 1, in <module>
app-1     |     from wagtail.documents.views.chooser import viewset as chooser_viewset
app-1     |   File "/venv/lib/python3.13/site-packages/wagtail/documents/views/chooser.py", line 9, in <module>
app-1     |     from wagtail.admin.views.generic.chooser import (
app-1     |     ...<7 lines>...
app-1     |     )
app-1     |   File "/venv/lib/python3.13/site-packages/wagtail/admin/views/generic/__init__.py", line 7, in <module>
app-1     |     from .history import HistoryView  # noqa: F401
app-1     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1     |   File "/venv/lib/python3.13/site-packages/wagtail/admin/views/generic/history.py", line 14, in <module>
app-1     |     from wagtail.admin.filters import (
app-1     |     ...<3 lines>...
app-1     |     )
app-1     |   File "/venv/lib/python3.13/site-packages/wagtail/admin/filters.py", line 9, in <module>
app-1     |     from wagtail.admin.models import popular_tags_for_model
app-1     |   File "/venv/lib/python3.13/site-packages/wagtail/admin/models.py", line 22, in <module>
app-1     |     class Admin(models.Model):
app-1     |     ...<4 lines>...
app-1     |             ]
app-1     |   File "/venv/lib/python3.13/site-packages/django/db/models/base.py", line 136, in __new__
app-1     |     raise RuntimeError(
app-1     |     ...<3 lines>...
app-1     |     )
app-1     | RuntimeError: Model class wagtail.admin.models.Admin doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.

using the default .env dist values. Turning Wagtail admin off makes no difference. So there's either some configuration step missing, or perhaps the .env.dist may need updating for the app to start? Not sure if that's documentation issue, or a corner case in the settings.

(Only make run-prod seems to work out of the box without crashing?)

EDIT: Opening mozmeao/springfield#921 for visibility if that's app issue.