How to predict future trends

[Jwenyi]

Hi there,

I'm a surgeon and wanted to use tabpfnts for prediction of time-series data. I'm not very familiar with AI. I would be very grateful if you could give some suggestions.

I read your interesting tutorial. However, I'm wondering how I could forecast the future trends?
I input data with covariates from 1990-2023 (one target, two covariates), I knew how to predict by splitting the training and test set, like using 1990-2000 to predict 2001-2023. But, how could I forecast time-series after 2023, like 2023-2040?

Best,
Wenyi

[liam-sbhoo]

Hi @Jwenyi

To do that, you'd just have to create a test set that contain 2023-2040 in the timestamps.

On the other hand, can I assume you have the covariates info for 2023-2040 then?

[Jwenyi]

Hi @liam-sbhoo
Thanks so much for your kind help! I tried running the model as you suggested, but I ran into one major issue: for the years 2023–2040 I don’t have covariate data available. From my understanding, TabPFNTS treats time series forecasting essentially as a regression problem, which makes me wonder:
Is it possible to generate future predictions using only the historical covariates and target values, without requiring explicit covariates for the future horizon?
Alternatively, if future covariates are mandatory, would it be reasonable to project them separately using simpler models (e.g., ARIMA or LSTM) and then feed these extrapolated values into TabPFNTS?
In medical forecasting, the standard practice is often to rely only on historical data and covariates (without needing external future projections of covariates), so I’m curious whether TabPFNTS is flexible enough to support this setup.
Thanks again for your time and guidance!

[liam-sbhoo]

@Jwenyi

No worries - happy to help!

Technically it is possible for TabPFN-TS to generate future predictions using only historical covariates. However, there lies a more fundamental problem.

Let's say the covariates contain important info about the target:

1. in this case, TabPFN would learn the underlying relationship between the inputs (timestamps and covariates) and the target values.
2. however, when TabPFN predicts on the test set and when the covariates are missing, it is missing the key info to make predictions, and therefore a suboptimal/less-informative prediction.

And yes, as you suggested, you could perhaps do projection on the covariates and later feed these extrapolated values in TabPFN-TS. In this setting, you are essentially doing a "nested" forecasting: you first forecast on the covariates, and then use the output and forecast on the target value - it might work (depending on how hard it is to forecast on the covariates), although I'm also concerned about the compounding of error here.

[Jwenyi]

Hi @liam-sbhoo

Many thanks for your detailed guidance, it’s been really helpful!

Based on your suggestions, could I confirm whether the pipeline should be understood as follows:

1. Without covariate projections:

• If we don’t project covariates into the future, should I simply create a test set (pandas DataFrame) with timestamps from 2023–2040, where the target column is set to NA? In this case, should the covariate columns be omitted altogether, or should they still be included with NA values?

2. With covariate projections: If we do project covariates, I have two related concerns:

• Could TabPFNTS itself be used to generate single-variable forecasts of covariates?
• If not, would it be appropriate to use simpler models (e.g., ETS or ARIMA) to produce covariate projections? My main concern is whether the potential benefits of including covariate forecasts would outweigh the risks of compounding error.

Thanks again for your time and for clarifying these points!

Best,
Wenyi

[liam-sbhoo]

Regarding (1):

should I simply create a test set (pandas DataFrame) with timestamps from 2023–2040, where the target column is set to NA?

Yup that's right! You could simply use the utility function (generate_test_x, see the demo) to do that.

In this case, should the covariate columns be omitted altogether, or should they still be included with NA values?

I would try both, since the inference should be pretty fast. Although my intuition tells me that omitting the covariate columns would give better and more tangible result.

---

Regarding (2):

Could TabPFNTS itself be used to generate single-variable forecasts of covariates?

Yes it can - it's actually the primary use case at the moment. In this setting, you are treating the covariate as the target value and perform forecast on it.

My main concern is whether the potential benefits of including covariate forecasts would outweigh the risks of compounding error.

Agreed!

---

[Jwenyi]

Regarding (1):

should I simply create a test set (pandas DataFrame) with timestamps from 2023–2040, where the target column is set to NA?

Yup that's right! You could simply use the utility function (generate_test_x, see the demo) to do that.

In this case, should the covariate columns be omitted altogether, or should they still be included with NA values?

I would try both, since the inference should be pretty fast. Although my intuition tells me that omitting the covariate columns would give better and more tangible result.

Regarding (2):

Could TabPFNTS itself be used to generate single-variable forecasts of covariates?

Yes it can - it's actually the primary use case at the moment. In this setting, you are treating the covariate as the target value and perform forecast on it.

My main concern is whether the potential benefits of including covariate forecasts would outweigh the risks of compounding error.

Agreed!

Many thanks!! I will do these and see if it works! Thanks again for your kind help!

[Jwenyi]

Hi @liam-sbhoo

Thanks so much for your previous suggestions!
I tried to directly predict future trends by using generate_test_x, but failed. I would be very grateful for any suggestions you may have.

I attached my input data and script below. Briefly, i use historical time-series with covariates as input, use generate_test_x to generate test df (target is NA, no covariate cols), and send them to TabPFN-TS predictor:

Input data:
TS_input.csv

my code:

from datasets import load_dataset
from tabpfn_time_series import TimeSeriesDataFrame
from tabpfn_time_series.data_preparation import to_gluonts_univariate, generate_test_X
from tabpfn_time_series import TabPFNTimeSeriesPredictor, TabPFNMode
import pandas as pd

cause_dat = pd.read_csv("TS_input.csv",header=0,index_col=[0,1])
cause_dat.index = pd.MultiIndex.from_tuples(
    [(item_id, pd.to_datetime(timestamp)) for item_id, timestamp in cause_dat.index],
    names=['item_id', 'timestamp']
)
cause_dat = cause_dat.sort_index(level=['item_id', 'timestamp'])
ts_input = TimeSeriesDataFrame(cause_dat.drop(columns = ['item']))
pred_len = 27
test_tsdf = generate_test_X(ts_input, pred_len)

predictor = TabPFNTimeSeriesPredictor(
    tabpfn_mode=TabPFNMode.LOCAL,
)
pred = predictor.predict(ts_input, test_tsdf)

Error:

ValueError                                Traceback (most recent call last)
Cell In[13], line 7
      1 from tabpfn_time_series import TabPFNTimeSeriesPredictor, TabPFNMode
      3 predictor = TabPFNTimeSeriesPredictor(
      4     tabpfn_mode=TabPFNMode.LOCAL,
      5 )
----> 7 pred = predictor.predict(ts_input, test_tsdf)

File c:\Users\QiFei\anaconda3\envs\TapPFNTS\lib\site-packages\tabpfn_time_series\predictor.py:47, in TabPFNTimeSeriesPredictor.predict(self, train_tsdf, test_tsdf)
     39 """
     40 Predict on each time series individually (local forecasting).
     41 """
     43 logger.info(
     44     f"Predicting {len(train_tsdf.item_ids)} time series with config{self.tabpfn_worker.config}"
     45 )
---> 47 return self.tabpfn_worker.predict(train_tsdf, test_tsdf)

File c:\Users\QiFei\anaconda3\envs\TapPFNTS\lib\site-packages\tabpfn_time_series\tabpfn_worker.py:196, in LocalTabPFN.predict(self, train_tsdf, test_tsdf)
    190 item_ids_chunks = np.array_split(
    191     np.random.permutation(train_tsdf.item_ids),
    192     min(total_num_workers, len(train_tsdf.item_ids)),
    193 )
    195 # Run predictions in parallel
--> 196 predictions = Parallel(n_jobs=len(item_ids_chunks), backend="loky")(
    197     delayed(self._prediction_routine_per_gpu)(
    198         train_tsdf.loc[chunk],
    199         test_tsdf.loc[chunk],
    200         gpu_id=i
    201         % torch.cuda.device_count(),  # Alternate between available GPUs
    202     )
    203     for i, chunk in enumerate(item_ids_chunks)
    204 )
    206 predictions = pd.concat(predictions)
    208 # Sort predictions according to original item_ids order

File c:\Users\QiFei\anaconda3\envs\TapPFNTS\lib\site-packages\joblib\parallel.py:2072, in Parallel.__call__(self, iterable)
   2066 # The first item from the output is blank, but it makes the interpreter
   2067 # progress until it enters the Try/Except block of the generator and
   2068 # reaches the first `yield` statement. This starts the asynchronous
   2069 # dispatch of the tasks to the workers.
   2070 next(output)
-> 2072 return output if self.return_generator else list(output)

File c:\Users\QiFei\anaconda3\envs\TapPFNTS\lib\site-packages\joblib\parallel.py:1682, in Parallel._get_outputs(self, iterator, pre_dispatch)
   1679     yield
   1681     with self._backend.retrieval_context():
-> 1682         yield from self._retrieve()
   1684 except GeneratorExit:
   1685     # The generator has been garbage collected before being fully
   1686     # consumed. This aborts the remaining tasks if possible and warn
   1687     # the user if necessary.
   1688     self._exception = True

File c:\Users\QiFei\anaconda3\envs\TapPFNTS\lib\site-packages\joblib\parallel.py:1784, in Parallel._retrieve(self)
   1778 while self._wait_retrieval():
   1779     # If the callback thread of a worker has signaled that its task
   1780     # triggered an exception, or if the retrieval loop has raised an
   1781     # exception (e.g. `GeneratorExit`), exit the loop and surface the
   1782     # worker traceback.
   1783     if self._aborting:
-> 1784         self._raise_error_fast()
   1785         break
   1787     nb_jobs = len(self._jobs)

File c:\Users\QiFei\anaconda3\envs\TapPFNTS\lib\site-packages\joblib\parallel.py:1859, in Parallel._raise_error_fast(self)
   1855 # If this error job exists, immediately raise the error by
   1856 # calling get_result. This job might not exists if abort has been
   1857 # called directly or if the generator is gc'ed.
   1858 if error_job is not None:
-> 1859     error_job.get_result(self.timeout)

File c:\Users\QiFei\anaconda3\envs\TapPFNTS\lib\site-packages\joblib\parallel.py:758, in BatchCompletionCallBack.get_result(self, timeout)
    752 backend = self.parallel._backend
    754 if backend.supports_retrieve_callback:
    755     # We assume that the result has already been retrieved by the
    756     # callback thread, and is stored internally. It's just waiting to
    757     # be returned.
--> 758     return self._return_or_raise()
    760 # For other backends, the main thread needs to run the retrieval step.
    761 try:

File c:\Users\QiFei\anaconda3\envs\TapPFNTS\lib\site-packages\joblib\parallel.py:773, in BatchCompletionCallBack._return_or_raise(self)
    771 try:
    772     if self.status == TASK_ERROR:
--> 773         raise self._result
    774     return self._result
    775 finally:

ValueError: at least one array or dtype is required