Skip to content

Commit a20605b

Browse files
AlexSkrypnykAlexSkrypnyk
committed
Split CircleCI configuration into multiple files with GitHub Actions-style naming.
- Created build-test-deploy.yml for main pipeline - Created database-nightly.yml for overnight DB refresh - Created update-dependencies.yml for Renovate automation - Created vortex-test-postbuild.yml for Vortex post-build tests - Created vortex-test-didi-fi.yml for DIDI file-to-image tests - Created vortex-test-didi-ii.yml for DIDI image-to-image tests - Added comprehensive documentation in README.md - Added workflow mapping and shared components documentation - Preserved all conditional markers for installer processing - Used GitHub Actions-style naming conventions throughout
1 parent 4bce547 commit a20605b

File tree

9 files changed

+1972
-1
lines changed

9 files changed

+1972
-1
lines changed

‎.circleci/README.md‎

Lines changed: 317 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,318 @@
1+
# CircleCI Multiple Configuration Files
2+
3+
This directory contains CircleCI configuration split into multiple files, each representing a separate pipeline with specific triggers and purposes.
4+
5+
## Overview
6+
7+
Vortex uses CircleCI's **multiple configuration files** feature to organize CI/CD workflows logically. This improves maintainability and makes it easier to understand each pipeline's purpose.
8+
9+
## Configuration Files
10+
11+
### Main Project Pipelines
12+
13+
#### 1. `build-test-deploy.yml`
14+
**Pipeline Name:** `Database, Build, Test and Deploy`
15+
16+
**Purpose:** Core build, test, and deployment workflow for the project.
17+
18+
**Triggers:**
19+
- Push to any branch
20+
- Push to any tag
21+
22+
**Jobs:**
23+
- `database` - Download and cache database
24+
- `build` - Build stack, lint code, run tests
25+
- `deploy` - Deploy to specific branches
26+
- `deploy-tags` - Deploy tagged releases
27+
28+
**Conditionals:**
29+
- `database` job: Conditional on `!PROVISION_TYPE_PROFILE`
30+
- `deploy` and `deploy-tags` jobs: Conditional on `DEPLOYMENT`
31+
32+
---
33+
34+
#### 2. `database-nightly.yml`
35+
**Pipeline Name:** `Database - Nightly refresh`
36+
37+
**Purpose:** Overnight database refresh and caching for next-day builds.
38+
39+
**Triggers:**
40+
- Schedule: `0 18 * * *` (6 PM UTC daily)
41+
- Branch: `develop` only
42+
43+
**Jobs:**
44+
- `database-nightly` - Fresh database download and cache
45+
46+
**Conditionals:**
47+
- Entire file: Conditional on `!PROVISION_TYPE_PROFILE`
48+
49+
---
50+
51+
#### 3. `update-dependencies.yml`
52+
**Pipeline Name:** `Update dependencies`
53+
54+
**Purpose:** Self-hosted Renovate for automated dependency updates.
55+
56+
**Triggers:**
57+
- Schedule: `5 11,23 * * *` (11:05 AM and 11:05 PM UTC daily)
58+
- Manual: Pipeline parameter `run_update_dependencies`
59+
- Branch: `develop` only
60+
61+
**Jobs:**
62+
- `update-dependencies` - Run Renovate bot
63+
64+
**Requirements:**
65+
- `RENOVATE_TOKEN` environment variable
66+
- `RENOVATE_REPOSITORIES` environment variable
67+
- `RENOVATE_GIT_AUTHOR` environment variable
68+
69+
**Conditionals:**
70+
- Entire file: Conditional on `DEPS_UPDATE_PROVIDER_CI`
71+
72+
---
73+
74+
### Vortex Development Pipelines
75+
76+
These pipelines are used for Vortex framework testing and will be removed during project installation.
77+
78+
#### 4. `vortex-test-postbuild.yml`
79+
**Pipeline Name:** `Vortex - Test (Post-build)`
80+
81+
**Purpose:** Vortex framework validation after main build.
82+
83+
**Triggers:**
84+
- Push to any branch
85+
- Push to any tag
86+
87+
**Jobs:**
88+
- `vortex-test-postbuild` - Run Vortex post-build tests
89+
90+
**Conditionals:**
91+
- Entire file: Conditional on `VORTEX_DEV`
92+
93+
---
94+
95+
#### 5. `vortex-test-didi-fi.yml`
96+
**Pipeline Name:** `Vortex - Test (DIDI from file)`
97+
98+
**Purpose:** Test database-in-image workflow with file source.
99+
100+
**Triggers:**
101+
- Push to any branch (implicit)
102+
103+
**Jobs:**
104+
- `vortex-test-didi-database-fi` - Create DB image from file
105+
- `vortex-test-didi-build-fi` - Build site with DIDI image
106+
107+
**Conditionals:**
108+
- Entire file: Conditional on `VORTEX_DEV`
109+
110+
---
111+
112+
#### 6. `vortex-test-didi-ii.yml`
113+
**Pipeline Name:** `Vortex - Test (DIDI from registry)`
114+
115+
**Purpose:** Test database-in-image workflow with registry source.
116+
117+
**Triggers:**
118+
- Push to any branch (implicit)
119+
120+
**Jobs:**
121+
- `vortex-test-didi-database-ii` - Create DB image from registry
122+
- `vortex-test-didi-build-ii` - Build site with DIDI image
123+
124+
**Conditionals:**
125+
- Entire file: Conditional on `VORTEX_DEV`
126+
127+
---
128+
129+
## Setting Up Pipelines in CircleCI
130+
131+
**Prerequisites:**
132+
- CircleCI account connected to GitHub
133+
- **CircleCI GitHub App integration** (required for multiple pipelines)
134+
- Existing project configured in CircleCI
135+
136+
### Configuration Steps
137+
138+
1. **Access CircleCI UI:**
139+
- Go to your project in CircleCI
140+
- Navigate to **Project Settings** → **Project Setup**
141+
142+
2. **Create Each Pipeline:**
143+
144+
For each configuration file, click **Add Pipeline** and configure:
145+
146+
| Config File | Pipeline Name | Config Path | Trigger |
147+
|-------------|--------------|-------------|---------|
148+
| build-test-deploy.yml | `Database, Build, Test and Deploy` | `.circleci/build-test-deploy.yml` | Push to all branches, all tags |
149+
| database-nightly.yml | `Database - Nightly refresh` | `.circleci/database-nightly.yml` | Schedule (`0 18 * * *` on `develop`) |
150+
| update-dependencies.yml | `Update dependencies` | `.circleci/update-dependencies.yml` | Schedule (`5 11,23 * * *` on `develop`) + Manual |
151+
| vortex-test-postbuild.yml | `Vortex - Test (Post-build)` | `.circleci/vortex-test-postbuild.yml` | Push to all branches, all tags |
152+
| vortex-test-didi-fi.yml | `Vortex - Test (DIDI from file)` | `.circleci/vortex-test-didi-fi.yml` | Push to all branches |
153+
| vortex-test-didi-ii.yml | `Vortex - Test (DIDI from registry)` | `.circleci/vortex-test-didi-ii.yml` | Push to all branches |
154+
155+
3. **Configure Environment Variables:**
156+
157+
Ensure the following project-level environment variables are set:
158+
159+
**Required for all pipelines:**
160+
- `VORTEX_CONTAINER_REGISTRY_USER` - Docker registry username
161+
- `VORTEX_CONTAINER_REGISTRY_PASS` - Docker registry password
162+
163+
**Required for database pipelines:**
164+
- Add SSH key for database downloads (fingerprint in config)
165+
166+
**Required for deployment pipelines:**
167+
- Add SSH key for deployments (fingerprint in config)
168+
169+
**Required for update-dependencies pipeline:**
170+
- `RENOVATE_TOKEN` - GitHub access token
171+
- `RENOVATE_REPOSITORIES` - Repository to run Renovate on
172+
- `RENOVATE_GIT_AUTHOR` - Author for Renovate commits
173+
174+
### Trigger Configuration Examples
175+
176+
**For scheduled pipelines:**
177+
- In CircleCI UI, select **Trigger** → **Schedule**
178+
- Set cron expression and target branch
179+
- Example: `0 18 * * *` on branch `develop`
180+
181+
**For manual triggers:**
182+
- Use pipeline parameters in config file
183+
- Trigger via CircleCI UI or API
184+
185+
**For push triggers:**
186+
- Select **Trigger** → **GitHub event**
187+
- Choose appropriate events (push, tag, pull request)
188+
189+
---
190+
191+
## File Organization
192+
193+
```
194+
.circleci/
195+
├── README.md # This file
196+
├── WORKFLOW_MAPPING.md # Workflow mapping documentation
197+
├── SHARED_COMPONENTS.md # Shared component analysis
198+
├── build-test-deploy.yml # Main pipeline
199+
├── database-nightly.yml # Nightly DB refresh
200+
├── update-dependencies.yml # Dependency updates
201+
├── vortex-test-postbuild.yml # Vortex post-build tests
202+
├── vortex-test-didi-fi.yml # Vortex DIDI from file tests
203+
└── vortex-test-didi-ii.yml # Vortex DIDI from registry tests
204+
```
205+
206+
---
207+
208+
## Naming Conventions
209+
210+
Files follow **GitHub Actions-style naming** for consistency:
211+
212+
- **Kebab-case:** All lowercase with hyphens (e.g., `build-test-deploy.yml`)
213+
- **Descriptive names:** Action-oriented naming (e.g., `update-dependencies.yml`)
214+
- **Vortex prefix:** Development workflows prefixed with `vortex-` (e.g., `vortex-test-*.yml`)
215+
216+
Pipeline names in CircleCI UI also follow this convention for easy identification.
217+
218+
---
219+
220+
## Shared Components
221+
222+
Each configuration file includes its own copy of required aliases and steps:
223+
224+
- `runner_config` - Shared runner container configuration
225+
- `step_setup_remote_docker` - Setup remote Docker
226+
- `step_process_codebase_for_ci` - Process docker-compose for CI
227+
- `load_variables_from_dotenv` - Load environment variables
228+
229+
This duplication ensures each config file is **self-contained and independent**.
230+
231+
---
232+
233+
## Conditional Markers
234+
235+
Vortex uses conditional markers for installer processing:
236+
237+
```yaml
238+
#;< MARKER_NAME
239+
# Content included only if MARKER_NAME is true
240+
#;> MARKER_NAME
241+
```
242+
243+
Common markers:
244+
- `!PROVISION_TYPE_PROFILE` - Exclude if profile-based provisioning
245+
- `DEPLOYMENT` - Include deployment jobs
246+
- `DEPS_UPDATE_PROVIDER_CI` - Include CI-based dependency updates
247+
- `VORTEX_DEV` - Vortex development features (removed in consumer projects)
248+
- `TOOL_*` - Include specific tool integrations (PHPCS, PHPStan, Behat, etc.)
249+
250+
---
251+
252+
## Consumer Project Installation
253+
254+
During Vortex installation:
255+
256+
1. **Vortex development files are removed:**
257+
- `vortex-test-postbuild.yml`
258+
- `vortex-test-didi-fi.yml`
259+
- `vortex-test-didi-ii.yml`
260+
261+
2. **Consumer project receives:**
262+
- `build-test-deploy.yml`
263+
- `database-nightly.yml` (if using database downloads)
264+
- `update-dependencies.yml` (if using CI-based dependency updates)
265+
266+
3. **Files are processed:**
267+
- Conditional markers are evaluated
268+
- Sections are included/excluded based on project configuration
269+
- Comments and markers are cleaned up
270+
271+
---
272+
273+
## Troubleshooting
274+
275+
### Pipeline Not Triggering
276+
277+
- **Check CircleCI UI:** Verify pipeline is configured correctly
278+
- **Check triggers:** Ensure trigger matches your event (push, schedule, etc.)
279+
- **Check branch filters:** Ensure your branch matches trigger configuration
280+
281+
### Pipeline Failing
282+
283+
- **Check environment variables:** Ensure all required variables are set
284+
- **Check SSH keys:** Verify fingerprints match configured keys
285+
- **Check logs:** Review CircleCI job logs for specific errors
286+
287+
### Cross-Pipeline Dependencies
288+
289+
- **Important:** CircleCI multiple pipelines **cannot depend on each other**
290+
- Each pipeline runs independently
291+
- If job dependencies are needed, keep jobs in the same config file
292+
293+
---
294+
295+
## Additional Resources
296+
297+
- **CircleCI Multiple Pipelines:** https://circleci.com/docs/set-up-multiple-configuration-files-for-a-project/
298+
- **CircleCI GitHub App:** https://circleci.com/docs/github-apps-integration/
299+
- **Vortex Documentation:** https://www.vortextemplate.com
300+
- **GitHub Actions (reference):** `.github/workflows/` directory
301+
302+
---
303+
304+
## Maintenance
305+
306+
When updating configurations:
307+
308+
1. **Maintain consistency:** Keep shared components identical across files
309+
2. **Update all files:** If changing runner version, update in all configs
310+
3. **Test changes:** Push to test branch and verify all pipelines run correctly
311+
4. **Document changes:** Update this README if adding/removing pipelines
312+
313+
---
314+
315+
## Custom Scripts
316+
1317
You may add custom scripts, which would run only in CI, to this directory and
2-
reference them from `config.yml` file.
318+
reference them from configuration files.

0 commit comments

Comments
 (0)