Skip to content

Commit 1dfcb45

Browse files
author
Luis
committed
Feat: Implement MySQL backup solution with automated scheduling, retention management, and restore capabilities
1 parent 13cd964 commit 1dfcb45

6 files changed

Lines changed: 467 additions & 0 deletions

File tree

‎docker-compose.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,28 @@ services:
4646
networks:
4747
- shared_network
4848

49+
mysql-backup:
50+
container_name: template-mysql-backup
51+
build:
52+
context: ./mysql-backup
53+
dockerfile: Dockerfile
54+
restart: always
55+
depends_on:
56+
- mysql
57+
environment:
58+
MYSQL_HOST: mysql
59+
MYSQL_PORT: 3306
60+
MYSQL_USER: root
61+
MYSQL_PASSWORD: root
62+
MYSQL_DATABASE: template_db
63+
BACKUP_SCHEDULE: "0 2 * * *" # Daily at 2 AM
64+
BACKUP_RETENTION_DAYS: 7 # Keep backups for 7 days
65+
BACKUP_ON_STARTUP: "true" # Create backup when container starts
66+
volumes:
67+
- ./volume/mysql_backups:/backups
68+
networks:
69+
- shared_network
70+
4971
adminer:
5072
container_name: template-adminer
5173
image: adminer

‎mysql-backup/Dockerfile‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
FROM alpine:3.19
2+
3+
# Install MySQL client, bash, and cron
4+
RUN apk add --no-cache \
5+
mysql-client \
6+
bash \
7+
dcron \
8+
tzdata
9+
10+
# Copy backup scripts
11+
COPY backup.sh /usr/local/bin/backup.sh
12+
COPY restore.sh /usr/local/bin/restore.sh
13+
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
14+
15+
# Make scripts executable
16+
RUN chmod +x /usr/local/bin/backup.sh \
17+
&& chmod +x /usr/local/bin/restore.sh \
18+
&& chmod +x /usr/local/bin/entrypoint.sh
19+
20+
# Create backup directory
21+
RUN mkdir -p /backups
22+
23+
# Set entrypoint
24+
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]

‎mysql-backup/README.md‎

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
# MySQL Auto Backup Setup
2+
3+
This directory contains the MySQL automatic backup solution for the FastAPI template project.
4+
5+
## Architecture
6+
7+
This backup solution uses a **lightweight Alpine Linux container** that only includes:
8+
- MySQL client tools (`mysqldump`, `mysql`) for backup/restore operations
9+
- Cron daemon for scheduling
10+
- Bash for scripting
11+
12+
**No MySQL server is included** - the backup container connects to your existing MySQL container over the Docker network. This approach:
13+
- Minimizes resource usage (Alpine image is ~10MB vs MySQL image ~600MB)
14+
- Reduces security surface area
15+
- Follows the single-responsibility principle
16+
- Allows independent scaling and management
17+
18+
## Features
19+
20+
- **Automatic scheduled backups** using cron
21+
- **Compressed backups** (gzip) to save disk space
22+
- **Backup rotation** with configurable retention period
23+
- **Easy restore** functionality
24+
- **Container-based** solution that runs alongside your MySQL container
25+
26+
## Configuration
27+
28+
The backup service can be configured through environment variables in `docker-compose.yaml`:
29+
30+
| Variable | Default | Description |
31+
|----------|---------|-------------|
32+
| `MYSQL_HOST` | mysql | MySQL server hostname |
33+
| `MYSQL_PORT` | 3306 | MySQL server port |
34+
| `MYSQL_USER` | root | MySQL username |
35+
| `MYSQL_PASSWORD` | root | MySQL password |
36+
| `MYSQL_DATABASE` | template_db | Database to backup |
37+
| `BACKUP_SCHEDULE` | 0 2 * * * | Cron schedule (default: 2 AM daily) |
38+
| `BACKUP_RETENTION_DAYS` | 7 | Number of days to keep backups |
39+
| `BACKUP_ON_STARTUP` | true | Create backup when container starts |
40+
41+
### Cron Schedule Format
42+
43+
The `BACKUP_SCHEDULE` uses standard cron format:
44+
```
45+
* * * * *
46+
│ │ │ │ │
47+
│ │ │ │ └─── Day of week (0-7, Sunday = 0 or 7)
48+
│ │ │ └───── Month (1-12)
49+
│ │ └─────── Day of month (1-31)
50+
│ └───────── Hour (0-23)
51+
└─────────── Minute (0-59)
52+
```
53+
54+
Examples:
55+
- `0 2 * * *` - Daily at 2:00 AM
56+
- `0 */6 * * *` - Every 6 hours
57+
- `30 3 * * 0` - Weekly on Sunday at 3:30 AM
58+
- `0 0 1 * *` - Monthly on the 1st at midnight
59+
60+
## Usage
61+
62+
### Starting the Backup Service
63+
64+
The backup service is included in your docker-compose setup:
65+
66+
```bash
67+
# Start all services including backup
68+
docker-compose up -d
69+
70+
# Start only the backup service
71+
docker-compose up -d mysql-backup
72+
```
73+
74+
### Manual Backup
75+
76+
To create a backup manually:
77+
78+
```bash
79+
docker exec template-mysql-backup /usr/local/bin/backup.sh
80+
```
81+
82+
### Viewing Backup Logs
83+
84+
```bash
85+
# View real-time logs
86+
docker logs -f template-mysql-backup
87+
88+
# View last 100 lines
89+
docker logs --tail 100 template-mysql-backup
90+
```
91+
92+
### Listing Backups
93+
94+
```bash
95+
# List all backups
96+
ls -lh ./volume/mysql_backups/
97+
98+
# From within container
99+
docker exec template-mysql-backup ls -lh /backups/
100+
```
101+
102+
### Restoring from Backup
103+
104+
#### Interactive Restore
105+
106+
```bash
107+
# This will list available backups and let you choose
108+
docker exec -it template-mysql-backup /usr/local/bin/restore.sh
109+
```
110+
111+
#### Direct Restore
112+
113+
```bash
114+
# Restore specific backup file
115+
docker exec template-mysql-backup /usr/local/bin/restore.sh /backups/mysql_backup_template_db_20240115_020000.sql.gz
116+
```
117+
118+
### Backup File Location
119+
120+
Backups are stored in: `./volume/mysql_backups/`
121+
122+
Backup files are named: `mysql_backup_<database>_<timestamp>.sql.gz`
123+
124+
## Backup Strategy Best Practices
125+
126+
1. **Test Restores Regularly**: Periodically test your restore process to ensure backups are valid
127+
2. **Monitor Disk Space**: Ensure sufficient disk space for backups
128+
3. **Off-site Backups**: Consider copying backups to external storage or cloud
129+
4. **Adjust Retention**: Balance between storage space and backup history needs
130+
131+
## Troubleshooting
132+
133+
### Backup Fails
134+
135+
Check logs:
136+
```bash
137+
docker logs template-mysql-backup
138+
```
139+
140+
Common issues:
141+
- MySQL credentials incorrect
142+
- MySQL container not ready
143+
- Insufficient disk space
144+
- Permission issues on backup directory
145+
146+
### Container Won't Start
147+
148+
Ensure the mysql-backup directory and scripts exist:
149+
```bash
150+
ls -la mysql-backup/
151+
```
152+
153+
### Restore Fails
154+
155+
- Ensure MySQL is running
156+
- Check credentials
157+
- Verify backup file integrity:
158+
```bash
159+
gunzip -t ./volume/mysql_backups/backup_file.sql.gz
160+
```
161+
162+
## Advanced Usage
163+
164+
### Backing Up to S3/MinIO
165+
166+
You can extend the backup script to upload to S3/MinIO after local backup:
167+
168+
```bash
169+
# Add to backup.sh after successful backup
170+
aws s3 cp "${BACKUP_DIR}/${BACKUP_FILENAME}" s3://your-bucket/mysql-backups/ --endpoint-url=http://minio:9000
171+
```
172+
173+
### Multiple Database Backup
174+
175+
Modify the backup script to loop through multiple databases:
176+
177+
```bash
178+
DATABASES="db1 db2 db3"
179+
for db in $DATABASES; do
180+
mysqldump ... --databases $db | gzip > backup_${db}_${TIMESTAMP}.sql.gz
181+
done
182+
```
183+
184+
### Notification on Backup Failure
185+
186+
Add notification logic to the backup script:
187+
188+
```bash
189+
if ! perform_backup; then
190+
# Send notification (email, Slack, etc.)
191+
curl -X POST https://hooks.slack.com/services/YOUR/WEBHOOK/URL \
192+
-H 'Content-type: application/json' \
193+
--data '{"text":"MySQL backup failed!"}'
194+
fi
195+
```

‎mysql-backup/backup.sh‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
#!/bin/bash
2+
3+
# MySQL backup script with rotation
4+
# This script creates compressed backups and maintains a specified number of backups
5+
6+
# Configuration
7+
BACKUP_DIR="/backups"
8+
MYSQL_HOST="${MYSQL_HOST:-mysql}"
9+
MYSQL_PORT="${MYSQL_PORT:-3306}"
10+
MYSQL_USER="${MYSQL_USER:-root}"
11+
MYSQL_PASSWORD="${MYSQL_PASSWORD:-root}"
12+
MYSQL_DATABASE="${MYSQL_DATABASE:-template_db}"
13+
BACKUP_RETENTION_DAYS="${BACKUP_RETENTION_DAYS:-7}"
14+
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
15+
BACKUP_FILENAME="mysql_backup_${MYSQL_DATABASE}_${TIMESTAMP}.sql.gz"
16+
17+
# Create backup directory if it doesn't exist
18+
mkdir -p "${BACKUP_DIR}"
19+
20+
# Function to log messages
21+
log() {
22+
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1"
23+
}
24+
25+
# Function to perform backup
26+
perform_backup() {
27+
log "Starting backup of database: ${MYSQL_DATABASE}"
28+
29+
# Perform the backup
30+
if mysqldump \
31+
-h "${MYSQL_HOST}" \
32+
-P "${MYSQL_PORT}" \
33+
-u "${MYSQL_USER}" \
34+
-p"${MYSQL_PASSWORD}" \
35+
--single-transaction \
36+
--routines \
37+
--triggers \
38+
--add-drop-database \
39+
--databases "${MYSQL_DATABASE}" | gzip > "${BACKUP_DIR}/${BACKUP_FILENAME}"; then
40+
41+
log "Backup completed successfully: ${BACKUP_FILENAME}"
42+
43+
# Get backup size
44+
BACKUP_SIZE=$(du -h "${BACKUP_DIR}/${BACKUP_FILENAME}" | cut -f1)
45+
log "Backup size: ${BACKUP_SIZE}"
46+
47+
return 0
48+
else
49+
log "ERROR: Backup failed!"
50+
return 1
51+
fi
52+
}
53+
54+
# Function to clean old backups
55+
cleanup_old_backups() {
56+
log "Cleaning up backups older than ${BACKUP_RETENTION_DAYS} days"
57+
58+
# Find and delete old backups
59+
find "${BACKUP_DIR}" -name "mysql_backup_${MYSQL_DATABASE}_*.sql.gz" -type f -mtime +${BACKUP_RETENTION_DAYS} -exec rm {} \; -exec echo "Deleted: {}" \;
60+
61+
# List remaining backups
62+
log "Current backups:"
63+
ls -lh "${BACKUP_DIR}"/mysql_backup_${MYSQL_DATABASE}_*.sql.gz 2>/dev/null || log "No backups found"
64+
}
65+
66+
# Main execution
67+
main() {
68+
log "=== MySQL Backup Script Started ==="
69+
70+
# Wait for MySQL to be ready (useful on container startup)
71+
until mysqladmin ping -h"${MYSQL_HOST}" -P"${MYSQL_PORT}" -u"${MYSQL_USER}" -p"${MYSQL_PASSWORD}" --silent; do
72+
log "Waiting for MySQL to be ready..."
73+
sleep 5
74+
done
75+
76+
# Perform backup
77+
if perform_backup; then
78+
# Clean up old backups only if current backup succeeded
79+
cleanup_old_backups
80+
log "=== Backup process completed successfully ==="
81+
exit 0
82+
else
83+
log "=== Backup process failed ==="
84+
exit 1
85+
fi
86+
}
87+
88+
# Run main function
89+
main

‎mysql-backup/entrypoint.sh‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
#!/bin/bash
2+
3+
# Entrypoint script for MySQL backup container
4+
# This script sets up cron jobs and keeps the container running
5+
6+
# Function to log messages
7+
log() {
8+
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1"
9+
}
10+
11+
# Set up environment variables for cron
12+
printenv | grep -E '^MYSQL_|^BACKUP_' > /etc/environment
13+
14+
# Create cron job based on BACKUP_SCHEDULE environment variable
15+
BACKUP_SCHEDULE="${BACKUP_SCHEDULE:-0 2 * * *}" # Default: 2 AM daily
16+
17+
# Create crontab entry
18+
echo "${BACKUP_SCHEDULE} /usr/local/bin/backup.sh >> /var/log/mysql-backup.log 2>&1" > /etc/crontabs/root
19+
20+
# Create log file
21+
touch /var/log/mysql-backup.log
22+
23+
# Start cron daemon in foreground mode
24+
crond -f -l 2 &
25+
CRON_PID=$!
26+
27+
log "=== MySQL Backup Container Started ==="
28+
log "Backup schedule: ${BACKUP_SCHEDULE}"
29+
log "Backup retention: ${BACKUP_RETENTION_DAYS:-7} days"
30+
log "Target database: ${MYSQL_DATABASE:-template_db}"
31+
32+
# Run initial backup if BACKUP_ON_STARTUP is set
33+
if [ "${BACKUP_ON_STARTUP}" = "true" ]; then
34+
log "Running initial backup..."
35+
/usr/local/bin/backup.sh
36+
fi
37+
38+
# Keep container running and tail the log
39+
tail -f /var/log/mysql-backup.log &
40+
41+
# Wait for cron daemon
42+
wait $CRON_PID

0 commit comments

Comments
 (0)