Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC: Merge actual server into actual #4139

Draft
wants to merge 396 commits into
base: master
Choose a base branch
from

Conversation

MikesGlitch
Copy link
Contributor

@MikesGlitch MikesGlitch commented Jan 12, 2025

Current issue: Move the github actions into this repo

This is a test branch

I'm using it to see what issues we could face by merging the repos together. This will be deleted once I've finished testing/documenting.

Steps I've taken:

git remote add --fetch old_server https://github.com/actualbudget/actual-server.git

git fetch old_server

git branch oldservermaster old_server/master

git checkout oldservermaster

<Manually delete node_modules files/anything that doesn't belong in server>

<Manually copy contents of repo into packages/sync-server (create folder for it) - including git workflows but not .git folder and not .yarn folder, or .yarnrc.yml. You should end up with root folder of .git and packages folder only>

git add .
git c -am "Moved actual-server into actual repo"
 git co master
git co -b merge-actual-server-into-actual
git merge --allow-unrelated-histories oldservermaster

Add the following to the root package.json

    "start:server": "yarn rebuild-node && yarn workspace actual-sync start",

Remove the following from the sync-server/package.json:

  "engines": {
    "node": ">=18.0.0"
  },
  "packageManager": "[email protected]"

Keep the version specifiers in the sync-server/package.json - we'll work out how we want to reference them later - if we choose to add workspace references it will require building the dependencies which is more overhead for users.

Delete node_modules folder and run yarn again:

yarn 
git add .
git c -am "updating yarn lock file for sync-server"

Update load-config.js to point to the new webroot (I added the two '../', '../' to account for the new folder structure):

  webRoot: path.join(
    projectRoot,
    '../',
    '../',
    'node_modules',
    '@actual-app',
    'web',
    'build',
  ),

Notes

  • If this was merged release notes would be merged also. Check that it's OK that release notes are merged (and nested under Actual on the docs website). If it's not alright, think of a new way to do release notes - maybe create new folders upcoming-releases/actual and upcoming-releases/actual-server to allow us to specify more granularly
  • Remember to remove .yarn folder and .yarnrc.yml from the packages/server. If it's not removed yarn will have issues finding packages and server wont start

To merge this - DO NOT SQUASH. We want to keep the commits. Use a merge commit (may need to relax the PR rules for this)


OLD:

Created a separate PR to prep for this - #4160:

  • Update eslint file to ignore the sync-server directory (until we fix the code)
  • Update the yarnrc.yml to include enableTransparentWorkspaces: false - this allows us to continue using npm packages when we still use workspaces

Added the following to yarnrc.yml to enable us to use npm packages when the package is also in the workspace:

enableTransparentWorkspaces: false

Jackenmen and others added 30 commits March 21, 2023 17:47
This changes the behavior of configuration loading when
ACTUAL_CONFIG_PATH env var is not specified. With this change, syntax
errors in config.json will now be reported if ACTUAL_CONFIG_PATH env var
is not specified and the app will not proceed. When the config.json file
is not present or cannot be read, the behavior remains the same.
…ints (#166)

Exposes IBAN in the API which is required for
actualbudget#766 to work reliably.
When working on something else, I noticed that `.test.js` files were not
running due to `jest.config.json` not including them. I went ahead and
re-enabled these tests to make sure that unit tests are actually being
run.
…lbudget#141)

Users in actualbudget#99 report that Actual in Docker runs on armv7 platforms,
although a bit sluggish. I confirmed that the base images for Debian and
Alpine support the linux/arm/v7 target and have added them to the
platform list in the GitHub Actions workflow.

At least one user confirms it works with the bullseye default
`Dockerfile`, but before merging it would be great if someone can
confirm it works with the Dockerfile.alpine image:

```
git clone https://github.com/jamesmortensen/actual-server.git
cd actual-server
git checkout armv7-image
docker build -t actual-server -f Dockerfile.alpine .
docker run -p 5006:5006 actual-server
```

---------

Co-authored-by: Jed Fox <[email protected]>
Update all Docker Hub references to new `actualbudget` organization from
`jlongster` personal account.
We're officially an org now!

A bit of markdown/yaml auto-formatting snuck in, too.

Closes actualbudget#364
Corresponding update for the docs site in
actualbudget/docs#144

Simultaneous to merging, we need to update our `DOCKER_HUB_*` GitHub
secrets in this repo.

---------

Co-authored-by: Jed Fox <[email protected]>
This will help people sort out configuration issues. Will open a PR to
the docs as well to guide people to troubleshoot using this!
GitHub CI log:
```
[linux/arm/v6 base 6/8] RUN yarn workspaces focus --all --production
204.6 ➤ YN0007: │ bcrypt@npm:5.1.0 must be built because it never has been before or the last one failed
204.6 ➤ YN0007: │ better-sqlite3@npm:8.2.0 must be built because it never has been before or the last one failed
...
[linux/arm/v7 base 6/8] RUN yarn workspaces focus --all --production
203.8 ➤ YN0007: │ bcrypt@npm:5.1.0 must be built because it never has been before or the last one failed
203.8 ➤ YN0007: │ better-sqlite3@npm:8.2.0 must be built because it never has been before or the last one failed
```

It seems that both armv6 and armv7 have the same issues with `bcrypt`
and `better-sqlite3` not being built. These packages are required to
build from source, luckily QEMU use armv7l for compiling.

Tested and working on RPi Zero W.

---------

Co-authored-by: Jed Fox <[email protected]>
Seems like we already added `tsc` to build the project, but we use the
wrong babel preset (Flow) instead of the specific TS one. This is only
used in testing to make Jest work (from what I can tell).
CodeQL keeps yelling at us about this… I’m not sure if the filter is
smart enough to use this rate limit middleware to remove the warnings,
but at least we will be setting a reasonable bound on attempts to crack
the server password.
Co-authored-by: Matiss Janis Aboltins <[email protected]>
…192)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…(#190)

Helps with actualbudget#919 by adding the `all` field wit both
pending and booked transactions to the output of
`getTransactionsWithBalance()` and, by extension, the
`/nordigen/transactions` endpoint.

I could alter the `getTransactions()` to return the `all` field as well
but I figured that keeping it such that it returns the output from
Nordigen API 1:1 might be better so I left it as is. If you don't agree,
let me know and I'll update this.
A small fix: returning JSON response instead of plain-text.

The frontend always expects a JSON response. So a tiny fix here..
Previously, the latest artifact list was requested unauthenticated using
`ADD "https://api.github.com/..." /tmp/artifacts.json`. While this works
locally, on GitHub’s servers it seems that the per-IP rate limit was
exceeded. There isn’t a way to get Docker to pass the `Authorization`
header that I know of, so this work has been moved to an external shell
script that pulls down the relevant data.
Web: actualbudget#1087
Server: actualbudget/actual-server#207
Docs: actualbudget/docs#179

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Remove the bulk in favour of links to our core docs.

---------

Co-authored-by: Jed Fox <[email protected]>
Copy link

netlify bot commented Jan 12, 2025

Deploy Preview for actualbudget failed.

Name Link
🔨 Latest commit 11e0212
🔍 Latest deploy log https://app.netlify.com/sites/actualbudget/deploys/6786364a24ff350008ada088

throw new Error(
'accountId and startDate must either both be arrays or both be strings',
);
}

Check failure

Code scanning / CodeQL

Use of externally-controlled format string High

Format string depends on a
user-provided value
.
throw new Error('accountId and startDate arrays must be the same length');
}

const earliestStartDate = Array.isArray(startDate)

Check failure

Code scanning / CodeQL

Use of externally-controlled format string High

Format string depends on a
user-provided value
.
syncVersion: syncFormatVersion,
name: name,
encryptMeta: encryptMeta,
owner:

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
.
res.status(400).send('Single file ID is required');
return;
}

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
.
// Source: https://expressjs.com/en/guide/error-handling.html
return next(err);
}
console.log(`Error on endpoint ${req.url}`, err.message, err.stack);

Check failure

Code scanning / CodeQL

Use of externally-controlled format string High

Format string depends on a
user-provided value
.
Copy link
Contributor

github-actions bot commented Jan 12, 2025

Bundle Stats — desktop-client

Hey there, this message comes from a GitHub action that helps you and reviewers to understand how these changes affect the size of this project's bundle.

As this PR is updated, I'll keep you updated on how the bundle size is impacted.

Total

Files count Total bundle size % Changed
18 → 21 5.92 MB → 5.92 MB (+976 B) +0.02%
Changeset
File Δ Size
locale/nb-NO.json 🆕 +19 B 0 B → 19 B
locale/pt-BR.json 🆕 +19 B 0 B → 19 B
locale/cy.json 🆕 +17 B 0 B → 17 B
src/i18n.ts 📈 +378 B (+20.31%) 1.82 kB → 2.19 kB
locale/fr.json 📈 +516 B (+4.00%) 12.6 kB → 13.11 kB
locale/en.json 📈 +27 B (+0.03%) 96.69 kB → 96.72 kB
View detailed bundle breakdown

Added

Asset File Size % Changed
static/js/nb-NO.js 0 B → 19 B (+19 B) -
static/js/pt-BR.js 0 B → 19 B (+19 B) -
static/js/cy.js 0 B → 17 B (+17 B) -

Removed

No assets were removed

Bigger

Asset File Size % Changed
static/js/fr.js 12.6 kB → 13.11 kB (+516 B) +4.00%
static/js/index.js 3.74 MB → 3.74 MB (+378 B) +0.01%
static/js/en.js 96.69 kB → 96.72 kB (+27 B) +0.03%

Smaller

No assets were smaller

Unchanged

Asset File Size % Changed
static/js/pl.js 17 B 0%
static/js/ta.js 17 B 0%
static/js/zh-Hant.js 10.13 kB 0%
static/js/es.js 3.45 kB 0%
static/js/indexeddb-main-thread-worker-e59fee74.js 13.5 kB 0%
static/js/workbox-window.prod.es5.js 5.69 kB 0%
static/js/de.js 4.59 kB 0%
static/js/resize-observer.js 18.37 kB 0%
static/js/uk.js 120.47 kB 0%
static/js/BackgroundImage.js 122.29 kB 0%
static/js/AppliedFilters.js 10.24 kB 0%
static/js/useAccountPreviewTransactions.js 1.63 kB 0%
static/js/narrow.js 84.3 kB 0%
static/js/wide.js 105.92 kB 0%
static/js/ReportRouter.js 1.58 MB 0%

Copy link
Contributor

github-actions bot commented Jan 12, 2025

Bundle Stats — loot-core

Hey there, this message comes from a GitHub action that helps you and reviewers to understand how these changes affect the size of this project's bundle.

As this PR is updated, I'll keep you updated on how the bundle size is impacted.

Total

Files count Total bundle size % Changed
1 1.33 MB 0%

Changeset

No files were changed

View detailed bundle breakdown

Added

No assets were added

Removed

No assets were removed

Bigger

No assets were bigger

Smaller

No assets were smaller

Unchanged

Asset File Size % Changed
kcab.worker.js 1.33 MB 0%

@MikesGlitch MikesGlitch marked this pull request as ready for review January 13, 2025 21:53
@MikesGlitch MikesGlitch marked this pull request as draft January 13, 2025 21:54
}

if (Array.isArray(accountId) != Array.isArray(startDate)) {
console.log(accountId, startDate);

Check failure

Code scanning / CodeQL

Use of externally-controlled format string

Format string depends on a [user-provided value](1).
);
}
if (Array.isArray(accountId) && accountId.length !== startDate.length) {
console.log(accountId, startDate);

Check failure

Code scanning / CodeQL

Use of externally-controlled format string

Format string depends on a [user-provided value](1).
}

try {
await fs.writeFile(getPathForUserFile(fileId), req.body);

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression

This path depends on a [user-provided value](1).
}

res.setHeader('Content-Disposition', `attachment;filename=${fileId}`);
res.sendFile(getPathForUserFile(fileId));

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression

This path depends on a [user-provided value](1).
// Source: https://expressjs.com/en/guide/error-handling.html
return next(err);
}
console.log(`Error on endpoint ${req.url}`, err.message, err.stack);

Check failure

Code scanning / CodeQL

Use of externally-controlled format string

Format string depends on a [user-provided value](1).
@MikesGlitch MikesGlitch changed the title Merge actual server into actual POC: Merge actual server into actual Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.