CatalystBERT-v1

SEC-filings test results

SEC-only combined-input baseline · 8-K, 6-K, 10-Q, and 10-K filings · 2016–2026. Combined means text plus structured context.

Impact significance ranks whether a reaction is large relative to expected volatility. Direction ranks positive versus negative returns among significant reactions. These are separate tasks.

Impact ranking is strongest at 2h and 1d; direction remains close to chance. The aggregate scores do not show calibration, confidence intervals, or subgroup performance.

Data & methodology

One example represents a filing–issuer pair. The SEC acceptance timestamp determines which return windows can be labeled; future prices create training targets and are never inference inputs.

Filing sources and timing

The SEC builder reads model-ready filing inputs, uses accepted_at as the event timestamp, and keeps only rows with a usable exact timestamp. Preferred securities, warrants, rights, and units are excluded by the security-name filter. The accession number identifies the filing. The four form types remain explicit event categories.

Intraday endpoints use the latest completed minute-bar close, rejecting endpoint prices more than five minutes stale. Daily volatility is the sample standard deviation of up to 20 prior daily stock-minus-SPY log returns, requiring at least 10 valid observations.

Connecting filing labels to market returns

Horizon minutes are [5, 30, 120, 390]. A scaled return of +1 means a positive abnormal move of one expected volatility, not a 1% price return. Missing or nonpositive expected volatility makes a target unavailable. SPY adjustment removes its observed log return; this is not a beta-adjusted asset-pricing alpha estimate.

Label quality and leakage controls

• Mask a horizon if another event for the same asset falls inside its reaction window.
• Mask 1d targets crossing a material price-adjustment factor change.
• By default, exclude 09:30–10:00 ET acceptances from impact training and mask horizons whose prior move reaches three expected volatilities.
• Carry a separate target mask per horizon. Missing targets are zero-filled for batching but excluded from losses and evaluation.
• Use chronological splits and an asset map derived only from final training rows. Unseen validation, test, and inference assets map to 0.

These controls reduce contamination; they do not prove causal attribution to a filing. Metrics shown here are reported aggregates from the sq-models SEC v6 baseline; this site does not rerun the evaluation.

Input and output tensors

The tables document the current combined-mode implementation associated with this v1 research. B is batch size; L is padded token length; F = 5 market features; the four horizon columns always follow [5m, 30m, 2h, 1d]. The reported experiment used batch size 1 and maximum L = 1,024. An archived checkpoint’s own configuration remains authoritative.

Text serialization

[TARGET] Issuer Company
[TIME] datetime
[TITLE] Filing title
[CONTENT] Selected filing content…

The prefix and filing body are tokenized as a pair. Truncation applies only to the content sequence, preserving issuer and title. SEC prefixes omit the ticker because the current ticker does not prove historical ticker validity. Four section tokens are added to the tokenizer and the encoder embedding vocabulary is resized accordingly. No token_type_ids tensor is passed.

Market-feature order

1. pre_event_abnormal_return_5m
2. pre_event_abnormal_return_30m
3. pre_event_abnormal_return_2h
4. pre_event_abnormal_return_1d
5. pre_event_volatility_20d

The 1d pre-event feature uses the prior market close; the SEC baseline uses exact acceptance timestamps, and its market context is restricted to observations before the event. The feature vector and its availability mask are concatenated to [B, 10] before the market encoder.

Forward computation can use mixed precision; the inference adapter explicitly casts outputs to float32 before sigmoid and magnitude decoding. Sigmoid scores are not established calibrated probabilities. Magnitude is an absolute reaction in volatility units, not a direct price forecast.

Training targets and losses

ModernBERT-large

A bidirectional, encoder-only Transformer used here to encode filing text, rather than generate text. ModernBERT-large has 395 million parameters and 28 layers, with a native context window up to 8,192 tokens. It uses rotary positional embeddings, alternating local/global attention, and unpadding; this implementation requests FlashAttention 2. Official model card.

The encoder configuration specifies a hidden width of 1,024. CatalystBERT mean-pools non-padding token embeddings and adds its own structured inputs and prediction heads. The experiment’s 1,024-token cap is substantially shorter than the encoder’s supported context window. These are distinct limits.

SEC experiment and limitations

catalystbert-sec-v6-combined-1x1024: 275,697 training, 121,561 validation, and 137,174 test examples. SEC-only 8-K, 6-K, 10-Q, and 10-K filings from 2016–2026. One epoch; batch size 1; eight gradient-accumulation steps; learning rate 2e-5; impact threshold 1.0; seed 17. One 1,024-token passage per example.

The reported results show impact-ranking signal, especially at 2h and 1d. Direction remains near chance; these results do not establish trading profitability, probability calibration, or causal attribution. SEC text-only and structured-only ablations are planned, not completed results.

The implementation also supports multiple passages with token tensors [B, P, L], a passage mask [B, P], and learned attention over passage representations. The reported baseline uses P = 1; it does not evaluate that extension.

News-only experiment

Earlier news-article baseline · catalystbert-expanded-v1-combined. These results are retained separately from the SEC-only model.

Impact significance ranks whether a reaction is large relative to expected volatility. Direction ranks positive versus negative returns among significant reactions. These are separate tasks.

Impact ranking is strongest at shorter horizons; direction remains near chance. The aggregate scores do not show calibration, confidence intervals, or subgroup performance.

News-only experiment · setup and limitations

catalystbert-expanded-v1-combined: 95,001 training, 30,445 validation, and 30,947 test examples. Three epochs; batch size 1; eight gradient-accumulation steps; learning rate 2e-5; seed 17. The checkpoint with lowest validation loss was evaluated on the chronological test split.

The earlier experiment uses news-article/target-company pairs. Its builder defaults to Benzinga, Finnhub, and Yahoo, with historical Google News opt-in. The current SEC methodology above is not a claim that this news evaluation used filing inputs.

These aggregate ablations suggest a small combined-input improvement in the documented experiment; no confidence interval or significance test is available. The model’s direction scores do not support a directional trading claim. Magnitude quality, calibration, and downstream strategy value are not established by the available results.