When creating a transaction I'm getting a 500 Server error

[kbmetz]

Pulled down a fresh version, made sure I erased all previous docker volumes and containers.

Every time I try to create a transaction I see the following error. But the transaction does get added if I'm in the 'Portfolio'. If I try to add the transaction from the 'Transactions' it doesn't get added at all.

OS: Ubuntu 22.04
Docker: 27.3.1
Uname -a :
Linux garage-01 6.8.0-48-generic #48~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct 7 11:24:13 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

[hackeresq]

Hey @kbmetz, can you run the following command on the host machine and share the last several entries in the log?

docker exec -it investbrain-app cat storage/logs/laravel.log

Appreciate your help reporting issues!

[kbmetz]

Happy to help! I've wanted something like this for a long time

I put the date from the machine at the bottom and also a successful curl command. As you can tell from the logs nothing is updating in regards to my stocks/transactions. Also, the dividend reported back already when the dividend hasn't actually been released yet.

[2024-11-07 22:11:00] production.WARNING: Failed calling method dividends (yahoo): Start date must be before end date
[2024-11-07 22:11:00] production.ERROR: Could not get market data: Start date must be before end date {"exception":"[object] (Exception(code: 0): Could not get market data: Start date must be before end date at /var/www/app/app/Interfaces/MarketData/FallbackInterface.php:40)
[stacktrace]
#0 /var/www/app/app/Models/Dividend.php(76): App\Interfaces\MarketData\FallbackInterface->__call('dividends', Array)
#1 /var/www/app/app/Console/Commands/RefreshDividendData.php(57): App\Models\Dividend::refreshDividendData('NEP')
#2 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): App\Console\Commands\RefreshDividendData->handle()
#3 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/Util.php(43): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#4 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(95): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#5 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#6 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/Container.php(690): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#7 /var/www/app/vendor/laravel/framework/src/Illuminate/Console/Command.php(213): Illuminate\Container\Container->call(Array)
#8 /var/www/app/vendor/symfony/console/Command/Command.php(279): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#9 /var/www/app/vendor/laravel/framework/src/Illuminate/Console/Command.php(182): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#10 /var/www/app/vendor/symfony/console/Application.php(1047): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /var/www/app/vendor/symfony/console/Application.php(316): Symfony\Component\Console\Application->doRunCommand(Object(App\Console\Commands\RefreshDividendData), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /var/www/app/vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(197): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(1203): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/app/artisan(13): Illuminate\Foundation\Application->handleCommand(Object(Symfony\Component\Console\Input\ArgvInput))
#16 {main}
"}
[2024-11-07 22:12:00] production.WARNING: Failed calling method quote (yahoo): cURL error 7: Failed to connect to fc.yahoo.com port 443 after 1 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://fc.yahoo.com
[2024-11-07 22:12:00] production.ERROR: Could not get market data: cURL error 7: Failed to connect to fc.yahoo.com port 443 after 1 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://fc.yahoo.com {"exception":"[object] (Exception(code: 0): Could not get market data: cURL error 7: Failed to connect to fc.yahoo.com port 443 after 1 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://fc.yahoo.com at /var/www/app/app/Interfaces/MarketData/FallbackInterface.php:40)
[stacktrace]
#0 /var/www/app/app/Models/MarketData.php(68): App\Interfaces\MarketData\FallbackInterface->__call('quote', Array)
#1 /var/www/app/app/Console/Commands/RefreshMarketData.php(58): App\Models\MarketData::getMarketData('PFE', false)
#2 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): App\Console\Commands\RefreshMarketData->handle()
#3 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/Util.php(43): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#4 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(95): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#5 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#6 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/Container.php(690): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#7 /var/www/app/vendor/laravel/framework/src/Illuminate/Console/Command.php(213): Illuminate\Container\Container->call(Array)
#8 /var/www/app/vendor/symfony/console/Command/Command.php(279): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#9 /var/www/app/vendor/laravel/framework/src/Illuminate/Console/Command.php(182): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#10 /var/www/app/vendor/symfony/console/Application.php(1047): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /var/www/app/vendor/symfony/console/Application.php(316): Symfony\Component\Console\Application->doRunCommand(Object(App\Console\Commands\RefreshMarketData), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /var/www/app/vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(197): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(1203): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/app/artisan(13): Illuminate\Foundation\Application->handleCommand(Object(Symfony\Component\Console\Input\ArgvInput))
#16 {main}
"}
[2024-11-07 22:12:00] production.WARNING: Failed calling method dividends (yahoo): Start date must be before end date
[2024-11-07 22:12:00] production.ERROR: Could not get market data: Start date must be before end date {"exception":"[object] (Exception(code: 0): Could not get market data: Start date must be before end date at /var/www/app/app/Interfaces/MarketData/FallbackInterface.php:40)
[stacktrace]
#0 /var/www/app/app/Models/Dividend.php(76): App\Interfaces\MarketData\FallbackInterface->__call('dividends', Array)
#1 /var/www/app/app/Console/Commands/RefreshDividendData.php(57): App\Models\Dividend::refreshDividendData('NEP')
#2 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): App\Console\Commands\RefreshDividendData->handle()
#3 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/Util.php(43): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#4 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(95): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#5 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#6 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/Container.php(690): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#7 /var/www/app/vendor/laravel/framework/src/Illuminate/Console/Command.php(213): Illuminate\Container\Container->call(Array)
#8 /var/www/app/vendor/symfony/console/Command/Command.php(279): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#9 /var/www/app/vendor/laravel/framework/src/Illuminate/Console/Command.php(182): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#10 /var/www/app/vendor/symfony/console/Application.php(1047): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /var/www/app/vendor/symfony/console/Application.php(316): Symfony\Component\Console\Application->doRunCommand(Object(App\Console\Commands\RefreshDividendData), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /var/www/app/vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(197): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(1203): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/app/artisan(13): Illuminate\Foundation\Application->handleCommand(Object(Symfony\Component\Console\Input\ArgvInput))
#16 {main}
"}
root@garage-01:/docker/investbrain$ date
Thu Nov 7 02:12:31 PM PST 2024
root@garage-01:/docker/investbrain$ curl fc.yahoo.com
curl: (7) Failed to connect to fc.yahoo.com port 80 after 3 ms: Connection refused
root@garage-01:~/docker/investbrain$ curl google.com

<TITLE>301 Moved</TITLE>

301 Moved

The document has moved here.

[hackeresq]

This is super helpful! Thank you for sharing (and for your patience).

I strongly suspect the "connection refused" error is the root cause. I have experienced Yahoo Finance being blocked on my pihole instance in the past.

Do you have pihole or some other adblock / firewall running? If so, can you check your query logs to see if that could be causing issues? Less likely, but Yahoo has been cracking down on these unofficial APIs and your IP range may be getting rate limited.

Separately, I was able to reproduce the experience you described on the "Transactions" page. It was a bug. That should be fixed on v1.0.2. You should be able to stop the stack and git pull the latest version of Investbrain (check the upgrade instructions).

[kbmetz]

Yep, I see Pihole blocking fc.yahoo.com. I've white listed it. Looks like because it's after market's end it's not updating now?

production.ERROR: Could not get market data: Start date must be before end date {"exception":"[object] (Exception(code: 0): Could not get market data: Start date must be before end date at /var/www/app/app/Interfaces/MarketData/FallbackInterface.php:40)

[hackeresq]

Fantastic. I'm definitely going to have to add a "troubleshooting" section to the readme. I imagine much of the self-hosted community uses pihole.

The market data refresh command runs every minute (but only actually refreshes when data is > 15 minutes old). So it should still be running right now.

Dumb question - do you have any future dated transactions?

[kbmetz]

As far as I know I don't have any future dated transactions except for the dividend. One of the stocks I have the dividend won't occur for another few weeks. But the screen already shows what the expected amount is. So maybe that's it?

[hackeresq]

Yeah, you nailed it! Yahoo is showing the dividend was issued yesterday.

I think I've spotted the issue:

$end_date = now();

if ( $dividends_meta->total_dividends ) {

    $start_date = $dividends_meta->last_date->addHours(48);
}

I'm adding 48 hours to the last dividend date for some reason that I can't quite remember. I'll have to clean this up.

Good news is, not really a bug - just unoptimized code. In theory everything should be working for you since the dividend refresh happens in the queue. Can you confirm?

[kbmetz]

That date on the 7th is the 'ex-date' or when you need to be holding the stock to get dividends (if I'm understanding correctly). For me, what I really want to know is the 'buy date' or when I actually get the dividend amount. What Yahoo is showing is only the 'expected' I believe.
If you look at https://finance.yahoo.com/quote/NEP/key-statistics/ you can see the Dividend date, which is when the money actually gets paid (or at least from my experience that's when it shows up in my trading account)

[hackeresq]

Yep, there are two dates associated with dividends.

Unfortunately, Yahoo only provides the Ex-Date. Some market data providers offer the issue date, and I've erred on using that date for the other providers.

I'm going to close this one as I believe your original issue has been resolved (the 500 error on creating transactions). If not, we can re-open!