TST, CI: new test failures in `test_regression_boston()` and `test_partial_fit_boston()`

Open
  • Is something wrong?
#145 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
57/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
github-actions, pandas, python, scikit-learn
Domain
ci-cd, testing

Research direction

Start with src/gfdl/tests/test_regression.py and inspect test_regression_boston() and test_partial_fit_boston(); reproduce the failure with the reported pytest command and pandas 3.1.0. Trace how the Boston dataset is loaded through scikit-learn’s fetch_openml and its ARFF parser to identify whether the incompatibility belongs in this project or upstream. Done when both tests pass in the affected CI environments with pandas 3.1.0.

Written by the indexing model from the issue text.

Description

continuous integration testing

These test failures are showing up in unrelated PRs in MacOS and Linux jobs. For example, in gh-129 and gh-143. Some sample logs for the failures are at:

I can reproduce locally on ARM Mac when I bump pandas from 3.0.6 to 3.1.0 and run i.e., python -m pytest src/gfdl/tests/test_regression.py::test_regression_boston on the latest main branch of our project at the time of writing. 3.1.0 was released six days ago.

Since GHA logs can disappear over time, I'll place a sample traceback below the fold as well.

____________________________________________________________________________ test_regression_boston[None-0.73452466] _____________________________________________________________________________

args = ('https://openml.org/data/v1/download/52643/boston.arff', '/Users/treddy/scikit_learn_data/openml')
kw = {'delay': 1.0, 'feature_names_to_select': ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', ...], 'md5_checksum': 'cdd361fb886627eaa80c92f90d0610cc', 'n_retries': 3, ...}
local_path = '/Users/treddy/scikit_learn_data/openml/openml.org/data/v1/download/52643/boston.arff.gz'

    @wraps(f)
    def wrapper(*args, **kw):
        try:
>           return f(*args, **kw)
                   ^^^^^^^^^^^^^^

../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_openml.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_openml.py:580: in _load_arff_response
    X, y, frame, categories = _open_url_and_load_gzip_file(
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_openml.py:568: in _open_url_and_load_gzip_file
    return load_arff_from_gzip_file(gzip_file, **arff_params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_arff_parser.py:533: in load_arff_from_gzip_file
    return _pandas_arff_parser(
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_arff_parser.py:447: in _pandas_arff_parser
    frame[col] = frame[col].cat.rename_categories(strip_single_quotes)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/pandas/core/accessor.py:128: in f
    return self._delegate_method(name, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/pandas/core/arrays/categorical.py:3253: in _delegate_method
    res = method(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/pandas/core/arrays/categorical.py:1431: in rename_categories
    new_categories = [new_categories(item) for item in self.categories]
                      ^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_arff_parser.py:435: in strip_single_quotes
    match = re.search(single_quote_pattern, input_string)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

pattern = re.compile("^'(?P<contents>.*)'$"), string = 0, flags = 0

    def search(pattern, string, flags=0):
        """Scan through string looking for a match to the pattern, returning
        a Match object, or None if no match was found."""
>       return _compile(pattern, flags).search(string)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       TypeError: expected string or bytes-like object, got 'int'

../../.local/share/uv/python/cpython-3.14.2-macos-aarch64-none/lib/python3.14/re/__init__.py:177: TypeError

During handling of the above exception, another exception occurred:

reg_alpha = None, expected = 0.73452466

    @pytest.mark.parametrize("reg_alpha, expected", [
        (0.1, 0.78550376),
        # NOTE: for Moore-Penrose, a large singular value
        # cutoff (rtol) is required to achieve reasonable R2 with
        # the Boston Housing dataset
        (None, 0.73452466),
    ])
    def test_regression_boston(reg_alpha, expected):
        # real-world data test with multi-layer RVFL
>       boston = fetch_openml(name="boston", version=1, as_frame=False)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

src/gfdl/tests/test_regression.py:120: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/utils/_param_validation.py:218: in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_openml.py:1161: in fetch_openml
    bunch = _download_data_to_bunch(
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_openml.py:715: in _download_data_to_bunch
    X, y, frame, categories = _retry_with_clean_cache(
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_openml.py:100: in wrapper
    return f(*args, **kw)
           ^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_openml.py:580: in _load_arff_response
    X, y, frame, categories = _open_url_and_load_gzip_file(
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_openml.py:568: in _open_url_and_load_gzip_file
    return load_arff_from_gzip_file(gzip_file, **arff_params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_arff_parser.py:533: in load_arff_from_gzip_file
    return _pandas_arff_parser(
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_arff_parser.py:447: in _pandas_arff_parser
    frame[col] = frame[col].cat.rename_categories(strip_single_quotes)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/pandas/core/accessor.py:128: in f
    return self._delegate_method(name, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/pandas/core/arrays/categorical.py:3253: in _delegate_method
    res = method(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/pandas/core/arrays/categorical.py:1431: in rename_categories
    new_categories = [new_categories(item) for item in self.categories]
                      ^^^^^^^^^^^^^^^^^^^^
../../python_venvs/py_314_rvfl/lib/python3.14/site-packages/sklearn/datasets/_arff_parser.py:435: in strip_single_quotes
    match = re.search(single_quote_pattern, input_string)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

pattern = re.compile("^'(?P<contents>.*)'$"), string = 0, flags = 0

    def search(pattern, string, flags=0):
        """Scan through string looking for a match to the pattern, returning
        a Match object, or None if no match was found."""
>       return _compile(pattern, flags).search(string)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       TypeError: expected string or bytes-like object, got 'int'

../../.local/share/uv/python/cpython-3.14.2-macos-aarch64-none/lib/python3.14/re/__init__.py:177: TypeError

This does not appear to be a connectivity issue related to one of the currently open issues. It may be that an upstream library needs to adapt to pandas changes, or that we do, I haven't checked in detail yet. Carefully crafted fixes/PRs welcome.

Dominant language
Python
Stars
3
Forks
6
Avg merge
1d 3h
Merged PRs (30d)
4

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from lanl/GFDL

All issues in lanl/GFDL

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.