Last updated on 2025-09-29 01:49:50 CEST.
| Flavor | Version | Tinstall | Tcheck | Ttotal | Status | Flags |
|---|---|---|---|---|---|---|
| r-devel-linux-x86_64-debian-clang | 1.2.1 | 7.58 | 204.35 | 211.93 | ERROR | |
| r-devel-linux-x86_64-debian-gcc | 1.2.1 | 5.66 | 141.16 | 146.82 | ERROR | |
| r-devel-linux-x86_64-fedora-clang | 1.2.1 | 328.87 | ERROR | |||
| r-devel-linux-x86_64-fedora-gcc | 1.2.1 | 316.68 | ERROR | |||
| r-devel-windows-x86_64 | 1.2.1 | 13.00 | 190.00 | 203.00 | ERROR | |
| r-patched-linux-x86_64 | 1.2.1 | 9.20 | 192.52 | 201.72 | ERROR | |
| r-release-linux-x86_64 | 1.2.1 | 8.42 | 192.86 | 201.28 | ERROR | |
| r-release-macos-arm64 | 1.2.1 | 111.00 | NOTE | |||
| r-release-macos-x86_64 | 1.2.1 | 213.00 | NOTE | |||
| r-release-windows-x86_64 | 1.2.1 | 12.00 | 191.00 | 203.00 | ERROR | |
| r-oldrel-macos-arm64 | 1.2.1 | 99.00 | NOTE | |||
| r-oldrel-macos-x86_64 | 1.2.1 | 192.00 | NOTE | |||
| r-oldrel-windows-x86_64 | 1.2.1 | 16.00 | 241.00 | 257.00 | ERROR |
Version: 1.2.1
Check: CRAN incoming feasibility
Result: NOTE
Maintainer: ‘Jakub Wiśniewski <jakwisn@gmail.com>’
The Description field contains
<arXiv:2104.00507>.
Please refer to arXiv e-prints via their arXiv DOI <doi:10.48550/arXiv.YYMM.NNNNN>.
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc
Version: 1.2.1
Check: Rd files
Result: NOTE
checkRd: (-1) choose_metric.Rd:35: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) choose_metric.Rd:36: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) choose_metric.Rd:37: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:23: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) disparate_impact_remover.Rd:28: Lost braces
28 | pigeonholing. The number of pigeonholes is fixed and equal to min{101, unique(a)}, where a is vector with values for subgroup. So if some subgroup is not numerous and
| ^
checkRd: (-1) fairness_check.Rd:47: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:48: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:49: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:50: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:51: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:52: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:53: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:54: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:55: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:56: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:57: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:58: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:61: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:62: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:63: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:64: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:65: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:66: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:12: Lost braces
12 | \item{scale}{logical, if code{TRUE} metrics will be scaled to mean 0 and sd 1. Default \code{FALSE}}
| ^
checkRd: (-1) fairness_heatmap.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:18: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_radar.Rd:18: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_radar.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:25: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:26: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:27: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:28: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:30: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:31: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:32: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:33: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) metric_scores.Rd:18: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) metric_scores.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:23: Lost braces in \itemize; \value handles \item{}{} directly
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc, r-devel-linux-x86_64-fedora-clang, r-devel-linux-x86_64-fedora-gcc, r-devel-windows-x86_64, r-patched-linux-x86_64, r-release-linux-x86_64, r-release-macos-arm64, r-release-macos-x86_64, r-release-windows-x86_64, r-oldrel-macos-arm64, r-oldrel-macos-x86_64, r-oldrel-windows-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 4.352454e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [38s/46s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1576007 , mean = 0.5452381 , max = 0.8659171
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8439043 , mean = -0.0003579772 , max = 0.7794764
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -43.79147 , mean = 744.6078 , max = 1609.777
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -278.5493 , mean = -1.033464e-12 , max = 331.3986
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -43.79147 , mean = 744.6078 , max = 1609.777
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -278.5493 , mean = -1.033464e-12 , max = 331.3986
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6345382
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-debian-clang
Version: 1.2.1
Check: re-building of vignette outputs
Result: ERROR
Error(s) in re-building vignettes:
...
--- re-building ‘Advanced_tutorial.Rmd’ using rmarkdown
[WARNING] Deprecated: --no-highlight. Use --syntax-highlighting=none instead.
--- finished re-building ‘Advanced_tutorial.Rmd’
--- re-building ‘Basic_tutorial.Rmd’ using rmarkdown
Quitting from Basic_tutorial.Rmd:254-257 [unnamed-chunk-19]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<error/rlang_error>
Error in `rep()`:
! attempt to replicate an object of type 'object'
---
Backtrace:
▆
1. ├─base::plot(fheatmap, text_size = 3)
2. └─fairmodels:::plot.fairness_heatmap(fheatmap, text_size = 3)
3. └─base::ifelse(...)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error: processing vignette 'Basic_tutorial.Rmd' failed with diagnostics:
attempt to replicate an object of type 'object'
--- failed re-building ‘Basic_tutorial.Rmd’
SUMMARY: processing the following file failed:
‘Basic_tutorial.Rmd’
Error: Vignette re-building failed.
Execution halted
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc, r-patched-linux-x86_64, r-release-linux-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [23s/27s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.166766 , mean = 0.5449216 , max = 0.8646708
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8468261 , mean = -4.147746e-05 , max = 0.7757302
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 25.82813 , mean = 747.611 , max = 1705.311
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -334.8318 , mean = -2.417907e-13 , max = 421.9965
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 25.82813 , mean = 747.611 , max = 1705.311
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -334.8318 , mean = -2.417907e-13 , max = 421.9965
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6444691
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-debian-gcc
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 1.280546e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.06718651 , mean = 0.6975836 , max = 0.9967857
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7225752 , mean = 0.002416363 , max = 0.634748
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-devel-linux-x86_64-fedora-clang
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [61s/119s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1553017 , mean = 0.5452801 , max = 0.865104
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8415599 , mean = -0.0003999694 , max = 0.7773532
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -24.42086 , mean = 739.7605 , max = 1613.883
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -460.8134 , mean = 1.050643e-13 , max = 281.0745
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -24.42086 , mean = 739.7605 , max = 1613.883
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -460.8134 , mean = 1.050643e-13 , max = 281.0745
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 6/13 metrics calculated for all models ( <1b>[33m7 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6330521
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 327.8173 , mean = 756.3617 , max = 1159.246
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -635.2394 , mean = 0.1289064 , max = 608.1891
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-fedora-clang
Version: 1.2.1
Check: re-building of vignette outputs
Result: ERROR
Error(s) in re-building vignettes:
--- re-building ‘Advanced_tutorial.Rmd’ using rmarkdown
[WARNING] Deprecated: --no-highlight. Use --syntax-highlighting=none instead.
--- finished re-building ‘Advanced_tutorial.Rmd’
--- re-building ‘Basic_tutorial.Rmd’ using rmarkdown
Quitting from Basic_tutorial.Rmd:254-257 [unnamed-chunk-19]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<error/rlang_error>
Error in `rep()`:
! attempt to replicate an object of type 'object'
---
Backtrace:
▆
1. ├─base::plot(fheatmap, text_size = 3)
2. └─fairmodels:::plot.fairness_heatmap(fheatmap, text_size = 3)
3. └─base::ifelse(...)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error: processing vignette 'Basic_tutorial.Rmd' failed with diagnostics:
attempt to replicate an object of type 'object'
--- failed re-building ‘Basic_tutorial.Rmd’
SUMMARY: processing the following file failed:
‘Basic_tutorial.Rmd’
Error: Vignette re-building failed.
Execution halted
Flavors: r-devel-linux-x86_64-fedora-clang, r-devel-linux-x86_64-fedora-gcc
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 1.280546e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-devel-linux-x86_64-fedora-gcc
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [56s/64s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1655739 , mean = 0.5448536 , max = 0.8678453
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8572962 , mean = 2.652539e-05 , max = 0.7792605
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -175.0127 , mean = 754.9539 , max = 1588.401
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -317.578 , mean = -1.331406e-12 , max = 286.1262
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -175.0127 , mean = 754.9539 , max = 1588.401
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -317.578 , mean = -1.331406e-12 , max = 286.1262
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 6/13 metrics calculated for all models ( <1b>[33m7 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6332758
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-fedora-gcc
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in 'fairmodels-Ex.R' failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 6.648002e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-devel-windows-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running 'testthat.R' [29s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1621235 , mean = 0.5451472 , max = 0.8663297
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8489144 , mean = -0.0002671006 , max = 0.7759765
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -109.8735 , mean = 744.6892 , max = 1555.709
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -287.9302 , mean = -1.068701e-15 , max = 302.1698
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -109.8735 , mean = 744.6892 , max = 1555.709
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -287.9302 , mean = -1.068701e-15 , max = 302.1698
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6456490
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at D:\RCompile\CRANpkg\local\4.6\fairmodels.Rcheck\tests\testthat\helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-windows-x86_64
Version: 1.2.1
Check: re-building of vignette outputs
Result: ERROR
Error(s) in re-building vignettes:
--- re-building 'Advanced_tutorial.Rmd' using rmarkdown
--- finished re-building 'Advanced_tutorial.Rmd'
--- re-building 'Basic_tutorial.Rmd' using rmarkdown
Quitting from Basic_tutorial.Rmd:254-257 [unnamed-chunk-19]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<error/rlang_error>
Error in `rep()`:
! attempt to replicate an object of type 'object'
---
Backtrace:
▆
1. ├─base::plot(fheatmap, text_size = 3)
2. └─fairmodels:::plot.fairness_heatmap(fheatmap, text_size = 3)
3. └─base::ifelse(...)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error: processing vignette 'Basic_tutorial.Rmd' failed with diagnostics:
attempt to replicate an object of type 'object'
--- failed re-building 'Basic_tutorial.Rmd'
SUMMARY: processing the following file failed:
'Basic_tutorial.Rmd'
Error: Vignette re-building failed.
Execution halted
Flavors: r-devel-windows-x86_64, r-release-windows-x86_64, r-oldrel-windows-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 4.352454e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavors: r-patched-linux-x86_64, r-release-linux-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [35s/41s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1479666 , mean = 0.5450806 , max = 0.8661756
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8494754 , mean = -0.0002005194 , max = 0.7860247
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -93.18112 , mean = 753.9177 , max = 1773.093
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -353.6407 , mean = -2.970336e-13 , max = 342.2659
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -93.18112 , mean = 753.9177 , max = 1773.093
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -353.6407 , mean = -2.970336e-13 , max = 342.2659
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6453402
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-patched-linux-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [35s/39s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1569783 , mean = 0.5447946 , max = 0.8597974
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8447711 , mean = 8.554848e-05 , max = 0.7832734
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -266.9008 , mean = 733.0387 , max = 1743.397
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -325.2918 , mean = 7.519649e-13 , max = 259.8435
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -266.9008 , mean = 733.0387 , max = 1743.397
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -325.2918 , mean = 7.519649e-13 , max = 259.8435
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6428837
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-release-linux-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in 'fairmodels-Ex.R' failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 6.648002e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-release-windows-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running 'testthat.R' [29s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1493564 , mean = 0.544765 , max = 0.8684608
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8554452 , mean = 0.0001150618 , max = 0.7846394
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -100.0618 , mean = 755.3238 , max = 1884.965
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -305.8971 , mean = 4.224009e-13 , max = 298.0199
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -100.0618 , mean = 755.3238 , max = 1884.965
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -305.8971 , mean = 4.224009e-13 , max = 298.0199
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 6/13 metrics calculated for all models ( <1b>[33m7 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6444568
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at D:\RCompile\CRANpkg\local\4.5\fairmodels.Rcheck\tests\testthat\helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-release-windows-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in 'fairmodels-Ex.R' failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 6.648002e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-oldrel-windows-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running 'testthat.R' [43s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1503749 , mean = 0.544201 , max = 0.8658834
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8513474 , mean = 0.0006791388 , max = 0.7885971
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -43.09465 , mean = 746.6783 , max = 1708.651
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -289.6368 , mean = 5.73353e-13 , max = 365.18
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -43.09465 , mean = 746.6783 , max = 1708.651
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -289.6368 , mean = 5.73353e-13 , max = 365.18
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6438229
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at D:\RCompile\CRANpkg\local\4.4\fairmodels.Rcheck\tests\testthat\helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-oldrel-windows-x86_64