catboost/catboost

JSON export incompletely specifies categorical splits of non-symmetric trees

Open

#2 414 ouverte le 21 juin 2023

Voir sur GitHub
 (7 commentaires) (0 réactions) (1 assigné)Python (1 163 forks)batch import
feature requestgood first issue

Métriques du dépôt

Stars
 (7 556 stars)
Métriques de merge PR
 (Aucune PR mergée en 30 j)

Description

Problem: JSON export incompletely specifies categorical splits (see below) catboost version: 1.2 Operating System: Mac OS X 13.3.1 CPU: Apple Silicon (M2) GPU: N/A

In the JSON export of a non-symmetric tree, the splits of categorical variables don't specify which categorical variable and prior are used for that split. Only the split_index is specified, and as far as I can tell, there is no way to use this to obtain the missing information.

Example:

import numpy as np
import pandas as pd
from catboost import CatBoostRegressor, Pool
import random

# Generate some random data
n_trees = 5
n_samples = 1000
n_float_features = 3
n_categorical_features = 2
n_cat_feature_values = 6

import string
def random_string():
    return ''.join(random.choices(string.ascii_uppercase, k=4))

cat_feature_values = [['feature_value_%i_' % j + random_string() for i in range(n_cat_feature_values)] for j in range(n_categorical_features)]

X = [list(np.random.rand(n_float_features)) + [random.choice(cat_feature_values[j]) for j in range(n_categorical_features)] for _ in range(n_samples)]
y = np.random.rand(len(X))

col_names = ['float_%02i' % i for i in range(n_float_features)] + ['categorical_%02i' % i for i in range(n_categorical_features)]
X_df = pd.DataFrame(X, columns=col_names)

cat_feature_names = [col for col in col_names if col.startswith('categorical_')]
                    
pool = Pool(
    X_df,
    label=y,
    cat_features=cat_feature_names,
    embedding_features=[]
)

model = CatBoostRegressor(
    iterations=n_trees,
    grow_policy='Depthwise',
)
model.fit(pool, eval_set=pool)

import json

fn = 'json-export-is-incomplete.json'
model.save_model(fn, format='json', pool=pool)
with open(fn) as f:
    json_data = json.load(f)

json_data['trees'][0]

This outputs

{'left': {'left': {'left': {'left': {'left': {'left': {'value': -0.0201874536772569,
       'weight': 3},
      'right': {'value': -0.035984302933017416, 'weight': 12},
      'split': {'border': 12.999999046325684,
       'ctr_target_border_idx': 0,
       'split_index': 127,
       'split_type': 'OnlineCtr'}},
     'right': {'left': {'value': 0, 'weight': 0},
      'right': {'value': -0.018084313720464706, 'weight': 1},
      'split': {'border': 2.9999990463256836,
       'ctr_target_border_idx': 0,
       'split_index': 80,
       'split_type': 'OnlineCtr'}},
     'split': {'border': 0.8814510107040405,
      'float_feature_index': 0,
      'split_index': 15,
      'split_type': 'FloatFeature'}},
...

Notice that splits of type FloatFeature have the float_feature_index. What is needed is the analog for splits of type OnlineCtr.

Guide contributeur