Skip to content

Latest commit

 

History

History
1921 lines (1812 loc) · 167 KB

CHANGES.md

File metadata and controls

1921 lines (1812 loc) · 167 KB
Raw

Changes

This file tracks changes to the raiwidgets package. It should be updated in PRs that make changes, not separately. At the top of the file will always be a v-next entry to add changes that were made since the last release. Each set of changes should be grouped by

  • educational materials
  • new features
  • breaking changes
  • bug fixes and tests
  • other

Note that it is not required to have an entry for every pull request. Instead, please try to add only changes that are meaningful to users who read this file to understand what changed.

v-next

  • educational materials
  • new features
  • breaking changes
  • bug fixes and tests
  • other

v0.32.0

  • new features

    • Responsible AI Dashboard

      • add socket implementation to RAI dashboard object detection and question answering metrics calls by @imatiach-msft in microsoft#2394
      • Dashboard Port Updates by @Advitya17 in microsoft#2406
  • bug fixes and tests

    • Responsible AI Dashboard

      • release rai-core-flask 0.7.0 by @imatiach-msft in microsoft#2398
      • update raiwidgets to rai-core-flask v0.7.0 by @imatiach-msft in microsoft#2401
      • Release new version of rai_core_flask by @Advitya17 in microsoft#2407
      • fallback to request if socket connection fails for object detection and question answering metrics for backcompat by @imatiach-msft in microsoft#2409
      • Side Panel Bug Fix by @Advitya17 in microsoft#2414

    • RAI Vision

      • OD Explanation Loader Fix by @Advitya17 in microsoft#2381
      • Optional support for IS_CROWD for OD COCO label format by @Advitya17 in microsoft#2395
      • Import & data loading fixes for IC & multilabel notebooks by @Advitya17 in microsoft#2410
      • fix RangeIndex error in error analysis for object detection models by @imatiach-msft in microsoft#2411
      • add average parameter to MeanAveragePrecision to specify micro or macro calculation by @imatiach-msft in microsoft#2412

    • Counterfactual

      • Update requirements.txt to upgrade dice-ml to 0.11.0 by @gaugup in microsoft#2397
      • Refer to predicted value from Counterfactual data by @gaugup in microsoft#2402

    • Error analysis

      • Accessibility fix for 'Error Analysis' header by @Advitya17 in microsoft#2415
  • other
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 107831016 by @RubyZ10 in microsoft#2393
    • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20231003114057604 to main by @csigs in microsoft#2384
    • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20231030114314322 to main by @csigs in microsoft#2400
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 109426168 by @RubyZ10 in microsoft#2408
    • fix build errors due to changed exception messages and UCI repository expired certificate by @imatiach-msft in microsoft#2413

v0.31.0

  • new features

    • Responsible AI Dashboard

      • Heatmap Support for Object Detection by @Advitya17 in microsoft#2303
      • Vision Explanation Dashboard render refactor by @Advitya17 in microsoft#2306
      • Single Container in Image Data Explorer for Object Detection by @Advitya17 in microsoft#2330
      • Error Labeling integration into Image Data Explorer for Agg Labels by @Advitya17 in microsoft#2334
      • Detection info inside Object Detection Flyout by @Advitya17 in microsoft#2339
      • Categorical Exif Metadata support for Vision Dashboard by @Advitya17 in microsoft#2348
      • Error Labeling Integration in Table View by @Advitya17 in microsoft#2350
      • Numerical Exif Metadata support for Image Dashboards by @Advitya17 in microsoft#2358
      • add local feature importance info button and include shap values in description for RAI text dashboard by @imatiach-msft in microsoft#2373
      • Error Labeling Integration in Class View by @Advitya17 in microsoft#2354
      • Sorting support for Object Detection labels by @Advitya17 in microsoft#2376
      • Forecasting: generate time series as cohorts by @romanlutz in microsoft#2355

    • RAI utils

      • Add pandas timestamp support to json serialization utility function in raiutils by @romanlutz in microsoft#2320

    • RAI tabular

      • Forecasting: consume locally served model by @romanlutz in microsoft#2319
  • bug fixes and tests

    • Responsible AI Dashboard

      • accessibility: add aria label to RAI text dashboard slider by @imatiach-msft in microsoft#2293
      • Added ariaLabel to cohort dropdown by @Advitya17 in microsoft#2296
      • UI Tests for Object Detection Widgets in Model Overview by @Advitya17 in microsoft#2295
      • move text UI individual feature importance tests by @imatiach-msft in microsoft#2299
      • RAI Text Dashboard a11y: make scrollable pane focusable and use label attribute on slider by @imatiach-msft in microsoft#2305
      • Heatmap e2e test by @Advitya17 in microsoft#2239
      • fix true y and pred y not appearing for QA text dashboard in table view by @imatiach-msft in microsoft#2336
      • fix failing object detection e2e jupyter notebook UI tests on linux by @imatiach-msft in microsoft#2341
      • Expanded Object Detection Flyout & Image Sizes by @Advitya17 in microsoft#2335
      • add model explanation data to test app for question answering example on squad dataset by @imatiach-msft in microsoft#2326
      • fix alignment of true and predicted labels in vision data explorer by @imatiach-msft in microsoft#2369
      • fix choose cohorts close or cancel updating feature cohorts selection in flyout by @imatiach-msft in microsoft#2366
      • remove duplicate text in feature importance graph of the RAI Text dashboard by @imatiach-msft in microsoft#2371
      • remove non-functional page size selector from RAI Vision dashboard table view by @imatiach-msft in microsoft#2370
      • Backend and OD Notebook Dataset fixes by @Advitya17 in microsoft#2360
      • Data Analysis Label Fix for OD by @Advitya17 in microsoft#2365
      • Table View Search Fix for Multilabel & OD by @Advitya17 in microsoft#2367
      • Enabling Heatmap test for Feature Cohort + MO test fixes by @Advitya17 in microsoft#2340

    • RAI Text

      • fix failing vision tests due to new RAI validation logic which doesn't allow non string dropped column names by @imatiach-msft in microsoft#2302
      • add text feature importance e2e unit tests for covid jupyter notebook by @imatiach-msft in microsoft#2323
      • add text feature importance e2e unit tests for dbpedia jupyter notebook by @imatiach-msft in microsoft#2313
      • fix error analysis for RAITextInsights displaying all incorrect instances for text labels by @imatiach-msft in microsoft#2325
      • add text feature importance e2e unit tests for blbooksgenre jupyter notebook by @imatiach-msft in microsoft#2327
      • fix question answering pipeline indexing error in shap due to varied token length after masking by @imatiach-msft in microsoft#2332
      • add RAITextInsights openai notebook example and tests by @imatiach-msft in microsoft#2345

    • RAI Vision

      • fix failing vision tests due to new RAI validation logic which doesn't allow non string dropped column names by @imatiach-msft in microsoft#2302
      • OD model type support in responsibleai_vision for EA by @Advitya17 in microsoft#2233
      • Test case for EA-OD flow by @Advitya17 in microsoft#2322
      • fix data analysis description and tooltips hidden for large screens by @imatiach-msft in microsoft#2368
      • fix flaky object detection vision notebook test by allowing range of feature cohorts to be specified by @imatiach-msft in microsoft#2374
      • Unit test for Exif support in the Image Dashboards by @Advitya17 in microsoft#2382

    • RAI Tabular

    • Error analysis

  • other
    • update responsibleai-text and responsibleai-vision packages to responsibleai 0.30.0 by @imatiach-msft in microsoft#2300
    • Update requirements-linting.txt to add flake8-all-not-strings by @gaugup in microsoft#2276
    • Release erroranalysis 0.5.1 by @Advitya17 in microsoft#2304
    • Release Error Analysis 0.5.2 by @Advitya17 in microsoft#2317
    • increase verbosity for gated tests to help diagnose failures by @imatiach-msft in microsoft#2301
    • release responsibleai-vision 0.3.0 and responsibleai-text 0.2.0 by @imatiach-msft in microsoft#2315
    • Release responsibleai_vision 0.3.1 by @Advitya17 in microsoft#2321
    • Add create_complex_regression_pipeline to sklearn_model_utils.py by @gaugup in microsoft#2318
    • Remove pins on flake8 linter packages by @gaugup in microsoft#2328
    • release responsibleai-text 0.2.1 by @imatiach-msft in microsoft#2337
    • causal_manager.py: Move user config validation checks from compute() to add() by @gaugup in microsoft#2342
    • Fix links on README.md by @jamesbchao in microsoft#2349
    • Update README.md by @jamesbchao in microsoft#2352
    • Remove unreachable branch in test_rai_insights_save_and_load_scenarios.py by @gaugup in microsoft#2343
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 106178211 by @RubyZ10 in microsoft#2375
    • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20230928114108584 to main by @csigs in microsoft#2372
    • Add python 3.11 for erroranalysis, raiutils and rai-test-utils by @gaugup in microsoft#2385
    • Add 3.11 support to rai-core-flask by @gaugup in microsoft#2386
    • fix failing builds due to pytorch dependencies by @imatiach-msft in microsoft#2389
    • pin responsibleai text and vision packages to 0.30.0 by @romanlutz in microsoft#2387

v0.30.0

  • educational materials

    • RAI Text

  • new features

    • Responsible AI Dashboard

      • [Question Answering] Cache support for Model Overview Metrics by @Advitya17 in microsoft#2166
      • Individual feature importance interpret QA - part1 by @vinuthakaranth in microsoft#2167
      • Individual feature importance interpret QA by @vinuthakaranth in microsoft#2186
      • Error Analysis UI & Template support for Object Detection by @Advitya17 in microsoft#2203
      • add multiclass statistics to text and tabular RAI dashboards by @imatiach-msft in microsoft#2222
      • Add forecasting to raiwidgets and add a forecasting notebook by @romanlutz in microsoft#2252

    • RAI Text

      • [Question Answering] Cache support for Model Overview Metrics by @Advitya17 in microsoft#2166
      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.29.0 by @imatiach-msft in microsoft#2172
      • add base values and importances for ending tokens to QA explanations by @imatiach-msft in microsoft#2178

    • RAI Vision

      • [Question Answering] Cache support for Model Overview Metrics by @Advitya17 in microsoft#2166
      • update responsibleai-text and responsibleai-vision packages to responsibleai 0.29.0 by @imatiach-msft in microsoft#2172
      • [Object Detection] Model Overview Cache Support Extension for torchmetrics by @Advitya17 in microsoft#2170
      • OD model type support in raiutils for EA by @Advitya17 in microsoft#2227
      • GPU support for OD metrics by @Advitya17 in microsoft#2266

    • RAI Tabular

      • OD model type support in responsibleai for EA by @Advitya17 in microsoft#2230

    • Error analysis

      • Error Analysis backend support for Object Detection by @Advitya17 in microsoft#2225
  • bug fixes and tests

    • Responsible AI Dashboard

      • ModelAssessmentVision UI Test Template by @Advitya17 in microsoft#2174
      • Multiclass Image Classification Notebook for UI testing by @Advitya17 in microsoft#2175
      • Image Classification Notebook for UI testing #2175 by @Advitya17 in microsoft#2176
      • Vision Data Analysis / Data Balance E2E Tests + Refactor by @Advitya17 in microsoft#2183
      • Vision Model Overview E2E Tests - Refactor by @Advitya17 in microsoft#2185
      • OD ID Plot Hotfix by @Advitya17 in microsoft#2190
      • Fix heatmap by removing erroneous color configuration by @romanlutz in microsoft#2201
      • Vision Data Explorer Table & Class View tests by @Advitya17 in microsoft#2220
      • Vision Data Explorer Image Explorer Tests by @Advitya17 in microsoft#2188
      • Enabled EA UI testing for OD by @Advitya17 in microsoft#2251
      • CanvasTools OD Bounding Box Fix by @Advitya17 in microsoft#2267
      • Add forecasting to raiwidgets and add a forecasting notebook by @romanlutz in microsoft#2252
      • add e2e UI notebook tests to DBPedia text classification notebook by @imatiach-msft in microsoft#2265
      • Endpoint Interruption Logic for Model Overview with OD by @Advitya17 in microsoft#2268
      • Endpoint Interruption Logic for Model Overview with QA by @Advitya17 in microsoft#2274
      • add e2e UI notebook tests to blbooksgenre text classification notebook by @imatiach-msft in microsoft#2273
      • add RAITextInsights question answering notebook example and tests by @imatiach-msft in microsoft#2279
      • add e2e UI notebook tests to covid19 event multilabel text classification notebook by @imatiach-msft in microsoft#2282

    • RAI Vision

      • Require pillow>=10.0.0, catch drise failure due to low mask_res or num_masks by @romanlutz in microsoft#2181
      • Add retry logic to image reader by @kicha0 in microsoft#2206
      • fix error for automl object detection models when initializing error analysis and optimize explanation execution by @imatiach-msft in microsoft#2245
      • fix for aggregate parameter passed to torchmetrics by @imatiach-msft in microsoft#2235
      • pass transforms for optimized automl model explanations and improve error logging to include inner exception by @imatiach-msft in microsoft#2269

    • RAI Text

      • add RAITextInsights DBPedia notebook example and tests by @imatiach-msft in microsoft#2204
      • add RAITextInsights covid19 multilabel classification notebook example and tests by @imatiach-msft in microsoft#2221
      • add RAITextInsights blbooksgenre binary classification notebook example and tests by @imatiach-msft in microsoft#2226
      • Raise usererror on prediction failure. by @kicha0 in microsoft#2248
      • add RAITextInsights question answering notebook example and tests by @imatiach-msft in microsoft#2279
      • fix question answering error when running pipeline model on cuda by @imatiach-msft in microsoft#2291
      • add e2e UI notebook tests to covid19 event multilabel text classification notebook by @imatiach-msft in microsoft#2282

    • RAI Tabular

      • Change ValueError to UserConfigValidationException in casual_manager.py by @gaugup in microsoft#2209
      • Add input type validation for FeatureMetadata class by @gaugup in microsoft#2218
      • Move missing value validations before unique value validations by @gaugup in microsoft#2228
      • Pass categorical features after dropping dropped features in CounterfactualManager by @gaugup in microsoft#2241
      • RAIInsights: prevent failures on optional methods & fix feature range formatting for timestamps by @romanlutz in microsoft#2258
      • fix RAI explanation error for dropped categorical features after serialization by @imatiach-msft in microsoft#2261
      • Tabular Only: Address missing values as per each manager in responsibleai by @gaugup in microsoft#2263
      • responsibleai: Add prediction wrapper to missing value tests by @gaugup in microsoft#2270

    • Error Analysis

      • Update text insights and error_analysis_manager to handle pd.DataFrame predictions and prediction with type of string by @tongyu-microsoft in microsoft#2179
      • Error Analysis testing for IC & multilabel by @Advitya17 in microsoft#2205
      • change erroranalysis valueerror caused by invalid model prediction shape to user error by @imatiach-msft in microsoft#2254

    • Fairness Dashboard

      • Use only Fairlearn v0.7.0 due to breaking changes in v0.9.0 and lack of Python 3.7 support in v0.8.0 by @romanlutz in microsoft#2194
  • other
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 99116202 by @RubyZ10 in microsoft#2177
    • release responsibleai-vision 0.2.4 by @imatiach-msft in microsoft#2192
    • fix responsibleai-vision pypi release script by installing pycocotools from conda by @imatiach-msft in microsoft#2197
    • fix build CI failing due to old pip-tools failing with latest pip by @imatiach-msft in microsoft#2200
    • release scikit-learn upper bound in responsibleai package by @imatiach-msft in microsoft#2173
    • improve notebook test logging and verbosity to diagnose timeouts by @imatiach-msft in microsoft#2202
    • Widget spacing for OD MO widgets by @Advitya17 in microsoft#2213
    • Update _NOTICE.md file by @romanlutz in microsoft#2231
    • LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20230811114026921 to main by @csigs in microsoft#2237
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 101597327 by @RubyZ10 in microsoft#2236
    • reduce PR execution time for tabular RAI dashboard changes by @imatiach-msft in microsoft#2238
    • Release new version of raiutils with OD support by @Advitya17 in microsoft#2244
    • Removed python 3.6 from python PR gate by @Advitya17 in microsoft#2249
    • fix ci-notebook-vision transient 6 hour CI timeouts by installing older pyarrow by @imatiach-msft in microsoft#2212
    • Update explainer_manager.py to change docs from MimicWrapper to MimicExplainer by @gaugup in microsoft#2240
    • refactor notebook examples into tabular, text and vision folders by @imatiach-msft in microsoft#2253
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 101886729 by @RubyZ10 in microsoft#2255
    • Added additional paths for vision gates by @Advitya17 in microsoft#2256
    • move image folder for tabular responsibleaidashboard under tabular folder by @imatiach-msft in microsoft#2257
    • release erroranalysis 0.4.5 by @imatiach-msft in microsoft#2262
    • fix CI errors due to new mlflow and pydantic dependencies by @imatiach-msft in microsoft#2259
    • update responsibleai and raiwidgets to erroranalysis 0.4.5 by @imatiach-msft in microsoft#2264
    • convert CI-python to use conda and install pytorch dependencies by @imatiach-msft in microsoft#2272
    • update responsibleai to interpret-community 0.30.0 by @imatiach-msft in microsoft#2275
    • Remove deprecated module cohort.py from raiwidgets by @gaugup in microsoft#2278
    • Turn off flag should_construct_pandas_query in PredictionsModelWrapperClassification by @gaugup in microsoft#2277
    • Update CODEOWNERS to include romanlutz for responsibleai by @romanlutz in microsoft#2280
    • OD model and data util functions in rai_test_utils by @Advitya17 in microsoft#2246
    • Release new version of rai test utils by @Advitya17 in microsoft#2284
    • Updated release gate for rai test utils to install OD dependencies by @Advitya17 in microsoft#2285
    • Add torch installation support for rai test utils release by @Advitya17 in microsoft#2286
    • Release new version of erroranalysis with OD support by @Advitya17 in microsoft#2288
    • remove vision e2e notebook tests from raiwidgets and responsibleai release workflow by @imatiach-msft in microsoft#2292
    • Rename release-rai.yml to release-rai-tabular.yml by @gaugup in microsoft#2283

v0.29.0

  • educational materials

    • Doc update for responsibleai_vision by @Advitya17 in microsoft#2164
    • [Object Detection] User Flows / Architectures by @Advitya17 in microsoft#2134
    • [Object Detection] DFD Documentation by @natalie-isak in microsoft#2138
    • [Object Detection] Explanation Documentation by @natalie-isak in microsoft#2136
    • [Object Detection] CanvasTools documentation by @Advitya17 in microsoft#2133
    • [Object Detection] Model Overview Object Detection Documentation by @Advitya17 in microsoft#2132

  • new features

    • Responsible AI Dashboard

      • [Object Detection] Cache support for Model Overview Metrics by @Advitya17 in microsoft#2158
      • [Image Explorer] CanvasTools Image Loading support for Object Detection by @Advitya17 in microsoft#2097
      • [Object Detection] [Image Explorer] Bounding Box Support with CanvasTools on Flyout by @Advitya17 in microsoft#2116

  • bug fixes and tests

    • Responsible AI Dashboard

      • update responsibleai-text and responsibleai-vision packages to latest responsibleai 0.28.0 release by @imatiach-msft in microsoft#2159
      • Robot page exception in DatasetUtils where jointDataset.metaDict[property] is undefined by @tongyu-microsoft in microsoft#2148
      • Remove console log by @RubyZ10 in microsoft#2125
      • Refactor- Add datasetCohorts to ModelAssessementContext && create datasetCohort && add compositeFilter in DatasetCohort by @RubyZ10 in microsoft#2119

    • RAIInsights

      • raise error instead of warning when a user has missing data and add c… by @hawestra in microsoft#2143
      • Add columns names to create_multiclass_classification_dataset by @gaugup in microsoft#2114
      • add validation target column type in the classification scenario by @hawestra in microsoft#2127
      • update econml to latest 0.14.1 release by @imatiach-msft in microsoft#2112
      • Update dice-ml to 0.10.0 by @gaugup in microsoft#2145

    • Causal

      • Fix individual causal plot not rendering for 5k points by @vinuthakaranth in microsoft#2113
      • Add loading spinner for causal policy tab by @gaugup in microsoft#2121

    • RAI Text

      • merge RAI Text repository to RAI toolbox by @imatiach-msft in microsoft#2118
      • add responsibleai-text package tests to build gate by @imatiach-msft in microsoft#2139
      • release responsibleai-text 0.1.4 by @imatiach-msft in microsoft#2163
      • add release scripts for responsibleai-text and responsibleai-vision pypi packages by @imatiach-msft in microsoft#2156

    • RAI Vision

      • add responsibleai-vision package to responsible-ai-toolbox by @imatiach-msft in microsoft#2135
      • add responsibleai-vision package to gated CI tests by @imatiach-msft in microsoft#2152
      • add retry logic to common vision utils to make tests more robust by @imatiach-msft in microsoft#2154
      • add device parameter to responsibleai_vision package and pass to model wrappers by @imatiach-msft in microsoft#2157
      • release responsibleai-vision 0.2.3 by @imatiach-msft in microsoft#2162

    • RAI Test Utils

      • Update rai_test_utils in erroranalysis and responsibleai to 0.3.0 by @gaugup in microsoft#2111
      • rai_test_utils: Convert all namespaces in __all__ list as strings by @gaugup in microsoft#2128

  • other

    • fix build failures due to lint errors for onelocbuild yml by @imatiach-msft in microsoft#2137
    • Skipping python 3.7 tests on MACOS by @gaugup in microsoft#2129
    • fix build failures due to new joblib 1.3.0 release failing on windows with python 3.7 by @imatiach-msft in microsoft#2155
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 97902894 by @RubyZ10 in microsoft#2160
    • Update the python version to 3.8 in release workflows by @gaugup in microsoft#2161
    • fix job names in release scripts by @imatiach-msft in microsoft#2165

v0.28.0

  • educational materials
    • Correct test command and add clarification on GPG Key to CONTRIBUTING.md by @ziqi-ma in microsoft#2082
  • new features

    • Responsible AI Dashboard

      • QA Scenario metrics template by @mehektulsyan in microsoft#2034
      • Reset ComboBox Option for Object Detection by @natalie-isak in microsoft#2049
      • Integrate Question answering metrics from python backend by @mehektulsyan in microsoft#2054
      • Display details of points and bubble for highcharts in full-screen mode by @vinuthakaranth in microsoft#2060
      • Show message to select bubble after bubble plot renders by @vinuthakaranth in microsoft#2070
      • Add dataset metadata to IDataset by @gaugup in microsoft#2071
      • enable confusion matrix for binary and multiclass text and vision RAI dashboards by @imatiach-msft in microsoft#2108
  • bug fixes and tests

    • Responsible AI Dashboard

      • remove QA statistics which were mistakenly added to multilabel metrics by @imatiach-msft in microsoft#2048
      • fix multilabel RAI Vision Dashboard table view labels squished by @imatiach-msft in microsoft#2057
      • [Big data] Few bug fixes and changes by @vinuthakaranth in microsoft#2056
      • fix outline bug and add contributors to readme by @hawestra in microsoft#2081
      • Fix background color not changing with theme switch by @hawestra in microsoft#2084
      • Revert "fix outline bug" by @gaugup in microsoft#2086
      • Fix table view multi-selection by @RubyZ10 in microsoft#2087
      • [Big Data] Fix create cohort, filter range for index and dataset by @vinuthakaranth in microsoft#2089
      • Fix background color and outline by @hawestra in microsoft#2092
      • [Big data] Add telemetry by @vinuthakaranth in microsoft#2093
      • Fix bug -"All data" cohort detection is based on localized strings by @RubyZ10 in microsoft#2094
      • Retain filter selections when editing filter by @RubyZ10 in microsoft#2096
      • Adding Class Name To OD Dropdown by @natalie-isak in microsoft#2098
      • Refactor cohortList to be less than 300 lines & add datasetCohorts to context by @RubyZ10 in microsoft#2103

    • RAIInsights

      • Use cohort filter constatns from erroranalysis/responsibleai in tests by @gaugup in microsoft#2047
      • fix build failures by pinning statsmodels to older version due to numpy incompatibility by @imatiach-msft in microsoft#2053
      • Raise warning in case the test data has missing values by @gaugup in microsoft#2065
      • Add user feedback to compute() function by @gaugup in microsoft#697
      • Remove un-needed requests dependency by @gaugup in microsoft#2083
      • Raise value error for min max computation of feature range by @kicha0 in microsoft#2072
      • Move common_utils.py into responsibleai_dashboard directory by @gaugup in microsoft#2095
      • Port over logic of load_adult_income_dataset from dice-ml to unblock gates by @gaugup in microsoft#2106
      • Fix dimension mismatch error when dropped_features contains cat_features by @tongyu-microsoft in microsoft#2099

    • Error Analysis

      • Prevent overwriting of dataset in erroranalysis by @gaugup in microsoft#2050
      • fix multilabel erroranalysis metrics dropdown appearing blank by @imatiach-msft in microsoft#2063
      • release erroranalysis 0.4.4 by @imatiach-msft in microsoft#2105
      • update responsibleai and raiwidgets to erroranalysis 0.4.4 by @imatiach-msft in microsoft#2107

    • RAI Utils

      • Referencing UserConfigValidationException from raiutils instead of responsibleai by @gaugup in microsoft#1932

    • RAI Test Utils

    • Interpret

      • Fix robot page exception on feature importance when dropped_feature is defined by @tongyu-microsoft in microsoft#2051
      • [Big data] Fix local explanations crash by @vinuthakaranth in microsoft#2068

  • other

    • Add pyproject.toml file for isort config by @gaugup in microsoft#2052
    • Remove flaky HomeBrew Install by @gaugup in microsoft#2058
    • Update codecov-action@v2 to codecov-action@v3 by @gaugup in microsoft#2062
    • Remove rai_core_flask from paths in CI-python.yml by @gaugup in microsoft#2059
    • Update python-linting.yml to remove isort parameters by @gaugup in microsoft#2073
    • Fix ci-notebook error on failing to retrieve superconduct data zip by @tongyu-microsoft in microsoft#2102

v0.27.0

  • educational materials
    • Update explanation-dashboard-employee-attrition.ipynb to remove deprecated iteritems() and use items() instead by @gaugup in microsoft#2030
  • new features

    • Responsible AI Dashboard

      • Drop Down For Object Detection by @natalie-isak in microsoft#2010
      • Renaming Data Characteristics Tab by @natalie-isak in microsoft#2032
      • Object Detection UX & Documentation updates by @Advitya17 in microsoft#2037
      • Ensuring backwards compat b/w OD scenario and other vision scenarios by @natalie-isak in microsoft#2039

    • Model Overview

      • [Model Overview] Object Detection Metrics support by @Advitya17 in microsoft#2025
  • bug fixes and tests

    • Responsible AI Dashboard

      • [Refactor] Add preBuiltCohort to datasetCohorts in modelAssessmentContext by @RubyZ10 in microsoft#2007
      • Fix bugs 1.Save and switch doesn't do anything in cohort edit panel 2.Unable to edit cohort filters in certain scenario by @RubyZ10 in microsoft#2040

    • RAIInsights

      • make train data optional on RAIBaseInsights class by @imatiach-msft in microsoft#2029
      • responsibleai: Add test case for dataset having boolean features by @gaugup in microsoft#2042

    • Error Analysis

      • fix indexing error when creating a filtered error analysis tree view with a dataset that contains categoricals by @imatiach-msft in microsoft#2026
      • add EBM and LightGBM with TreeShap methods to calculate error correlation in error analysis by @imatiach-msft in microsoft#2018
      • release erroranalysis v0.4.3 by @imatiach-msft in microsoft#2043
      • update responsibleai and raiwidgets to erroranalysis 0.4.3 by @imatiach-msft in microsoft#2045
      • fix transient test failures by increasing timeout for gbm shap method by @imatiach-msft in microsoft#2044

    • RAI Utils

      • Move import of cohort module from raiwidgets to raituils by @gaugup in microsoft#2027

    • Interpret

      • update responsibleai to interpret-community 0.29.0 by @imatiach-msft in microsoft#2031

  • other

    • Add python 3.10 tests by @gaugup in microsoft#2033
    • fix failing builds by upgrading pip-tools after latest pip release by @imatiach-msft in microsoft#2038
    • Add 3.10 classifier for python packages by @gaugup in microsoft#2035
    • Upgrade python from 3.7 to 3.10 in few workflows by @gaugup in microsoft#2041

v0.26.0

  • educational materials
    • Update notebooks for categorical features to pass through FeatureMetadata by @tongyu-microsoft in microsoft#2011
  • new features

    • Responsible AI Dashboard

      • Adding Object Detection Fridge Data by @natalie-isak in microsoft#1998

    • RAIInsights

    • Model Overview

      • [Model Overview] Metrics Template support with OD-specific optional args by @Advitya17 in microsoft#2002
      • Object Detection Model Overview Flask template by @Advitya17 in microsoft#2004
      • Model Overview: Object Detection Widgets for Aggregate Methods, Class selection, & IoU Threshold by @Advitya17 in microsoft#1997
  • bug fixes and tests

    • Responsible AI Dashboard

      • Forecasting: UI code adjustments to renamed datetime_features and time_series_id_features by @romanlutz in microsoft#1993
      • [Refactor]1.Add DatasetCohort 2. Move FeatureFlights to core by @RubyZ10 in microsoft#2003
      • [Refactor]Add columnRanges and modelType to ModelAssessmentContext by @RubyZ10 in microsoft#2006

    • RAIInsights

      • Add validations for model predictions by @gaugup in microsoft#2008
      • Add validations if target column is included in FeatureMetadata by @gaugup in microsoft#2013
      • try pickling model instead of failing based on state functions and change to UserConfigValidationException by @imatiach-msft in microsoft#2014
      • update dependencies for rai-core-flask, raiwidgets and responsibleai packages by @imatiach-msft in microsoft#2016
      • Fix deprecated DataFrame indexer type in Error Analysis package by @JarvisG495 in microsoft#2019

    • Error Analysis

      • fix bug in tree view where slider params are not used by @imatiach-msft in microsoft#1990
      • fix error analysis showing float values with many zeros for some classification task nodes by @imatiach-msft in microsoft#1991
      • release erroranalysis v0.4.2 by @imatiach-msft in microsoft#2000
      • update raiwidgets and responsibleai to erroranalysis 0.4.2 by @imatiach-msft in microsoft#2001

v0.25.0

  • educational materials
    • Update README badges by @romanlutz in microsoft#1933
    • DOC improving several error strings by @romanlutz in microsoft#1943
    • Add rai_test_utils badge in README.md by @gaugup in microsoft#1953
    • improve installation instructions for onboarding to RAI toolbox repository by @imatiach-msft in microsoft#1962
    • fix outdated example link in Error Analysis README by @JarvisG495 in microsoft#1965
    • Update example notebook links in README docs by @JarvisG495 in microsoft#1970
    • Create README.md for python tests by @gaugup in microsoft#1984
  • new features

    • Responsible AI Dashboard

      • Forecasting RAI dashboard: expand test data, add categorical what if by @romanlutz in microsoft#1921

    • Dataset Explorer

      • [Large data Data explorer] Add individual data points - data explorer for large data. by @vinuthakaranth in microsoft#1919
      • add missing true and predicted labels to axis config for dataset explorer in multilabel text and vision scenarios by @imatiach-msft in microsoft#1960

    • Counterfactual

      • [Large data counterfactuals] Use data from local counterfactuals to display in counterfactuals panel by @vinuthakaranth in microsoft#1913

    • Interpret

      • [Large data feature importance] Add individual feature importance by @vinuthakaranth in microsoft#1929
  • bug fixes and tests

    • Responsible AI Dashboard

      • Fix typo in variable name disaggregatedAnalysisBaseCohortDislaimer by @gaugup in microsoft#1855
      • add real model for dev env to have better test data by @xuke444 in microsoft#1893
      • Remove setstate() out of render function in RegressionDistributionChart.tsx by @gaugup in microsoft#1922
      • Unify responsibleai constants by @romanlutz in microsoft#1925
      • Fix bubble chart render issue due to exception in SidePanel by @gaugup in microsoft#1941
      • Fix table view feature value && scroll bar by @RubyZ10 in microsoft#1954
      • [Large counterfactuals & explanations] Do not call local counterfactuals/explanations on unselecting the point by @vinuthakaranth in microsoft#1957
      • [Large data] Map classification outcome to string values, dither, count changes by @vinuthakaranth in microsoft#1955
      • add (none) category for empty predicted or true labels in multilabel vision dashboard for data characteristics by @imatiach-msft in microsoft#1961
      • fix CI notebook build via updated yarn command flags by @imatiach-msft in https://github.com/microsoft/responsible-ai-toolbox/pull/
      • upgrade tslib to 2.5.0 by @imatiach-msft in microsoft#1973
      • Forecasting: improve description in no-features case by @romanlutz in microsoft#1978
      • Forecasting: fix dropdown scroll issue by @romanlutz in microsoft#1979
      • Forecasting: restrict transformation names to 50 chars by @romanlutz in microsoft#1981
      • Forecasting: create a better UX for value inputs by @romanlutz in microsoft#1985
      • Forecasting: turn off Error Analysis by @romanlutz in microsoft#1987
      • Forecasting: fix button text to say "switch time series" by @romanlutz in microsoft#1988
      • Forecasting: validate characters for name field in transformation creation dialog by @romanlutz in microsoft#1986

    • RAIInsights

      • raiwidgets tests: Change post_data from tuple to list by @gaugup in microsoft#1920
      • Remove duplicate load() and save() for large data by @gaugup in microsoft#1930
      • Deprecate categorical_features argument in favor of feature_metadata.categorical_features, improve metadata validation, and add forecasting-related metadata by @romanlutz in microsoft#1934
      • Add number of samples in large data in meta.json by @gaugup in microsoft#1976

    • Data balance

      • [Bugfix] FeatureBalanceMeasures: skip computing measures for col if unique vals < 2 by @ms-kashyap in microsoft#1980

    • Model Overview

      • fix mean absolute error metric calculation in model overview panel by @imatiach-msft in microsoft#1950
      • Fix selection rate by @romanlutz in microsoft#1958

    • Dataset Explorer

      • Dataset explorer: Propagate number of bins to SDK backend API by @gaugup in microsoft#1940
      • Fix robot page exception on data explorer by @tongyu-microsoft in microsoft#1963
      • Add spline chart data fetch from SDK backend by @gaugup in microsoft#1959

    • Error Analysis

      • erroranalysis: Fix the scenario when samples in dataset and string_indexed_data don't match by @gaugup in microsoft#1915
      • improve error message for invalid features passed to matrix and tree views from API endpoint in dashboard by @imatiach-msft in microsoft#1917
      • improve erroranalysis error message when invalid features passed to matrix and tree methods by @imatiach-msft in microsoft#1918
      • Fix confusion matrix Heatmap bugs by @RubyZ10 in microsoft#1926
      • Fix focus indicator is not visible on all four sides for “Tree map” and "heat map" tabs. by @vinuthakaranth in microsoft#1947
      • release erroranalysis v0.4.1 by @imatiach-msft in microsoft#1966
      • update raiwidgets and responsibleai to erroranalysis 0.4.1 by @imatiach-msft in microsoft#1969

    • Interpret

      • Adding a limit on the number of samples used for computing global explanations by @gaugup in microsoft#1924

    • Causal

      • Move sort operation of out render() in CausalAggregateView by @gaugup in microsoft#1927
      • [Large data individual causal] Add individual causal what-if to large data by @vinuthakaranth in microsoft#1942

    • RAI Utils

    • RAI Test Utils

      • Migrate responsibleai tests to use utilities in rai_test_utils - Part 1 by @gaugup in microsoft#1937
      • Add more test helper functions in rai_test_utils by @gaugup in microsoft#1938
      • Create 0.1.0 release for rai_test_utils by @gaugup in microsoft#1952
      • Migrate responsibleai tests to use utilities in rai_test_utils - Part 2 by @gaugup in microsoft#1956

  • other

    • Update workflow ymls to pin pip-tools to below 6.12.2 by @gaugup in microsoft#1931

v0.24.1

  • educational materials
    • Update some sample notebooks to include dropped_features by @tongyu-microsoft in microsoft#1911
  • new features

    • Responsible AI Dashboard

      • Add forecasting dashboard - UI - time series selection & forecast comparison by @romanlutz in microsoft#1894
      • Confusion Matrix by @t-wangjohn in microsoft#1794
      • Update release workflows to include forecasting npm package by @tongyu-microsoft in microsoft#1901
      • Add forecasting dashboard - UI - transformation creation, table, and comparison by @romanlutz in microsoft#1874

    • Counterfactual

      • [Big data] Counterfactuals - Part2 by @vinuthakaranth in microsoft#1832
  • bug fixes and tests

    • Responsible AI Dashboard

      • Remove

        tags and replace with by @romanlutz in microsoft#1900

      • raiwidgets: Fix url in setup.py by @romanlutz in microsoft#1906
      • Enable logarithmic scaling, reflect cohort change and add add Revert to bubble chart button by @vinuthakaranth in microsoft#1905

    • RAIInsights

      • Move import to top level from within the function by @gaugup in microsoft#1898
      • Raise UserConfigValidationException when treatment_feature is empty list by @gaugup in microsoft#1904

    • Data balance

      • Make DataBalanceManager.compute() throw warning instead of exception by @ms-kashyap in microsoft#1902

    • Error Analysis

      • fix small data erroring out on mutual info score for error analysis guidance by @imatiach-msft in microsoft#1907
      • release erroranalysis v0.4.0 by @imatiach-msft in microsoft#1909
      • update raiwidgets and responsibleai to erroranalysis 0.4.0 by @imatiach-msft in microsoft#1910
      • Fix model wrapper to identify model type, remove task type from error analysis by @tongyu-microsoft in microsoft#1912

    • Interpret

      • Fix unwrap error in individual feature importance by @hawestra in microsoft#1897

  • other

    • Exempt only files from max-lines rule that are already in violation, but not new files (part 2/2) by @romanlutz in microsoft#1899

v0.24.0

  • educational materials
    • Add more linting instructions to CONTRIBUTING.md by @gaugup in microsoft#1810
    • Update some sample notebooks to include dropped_features by @tongyu-microsoft in microsoft#1836
    • Fix notebook links and typos by @gaugup in microsoft#1863
  • new features

    • Responsible AI Dashboard

      • add binary classification support for RAI text and vision scenarios by @imatiach-msft in microsoft#1778
      • add multilabel support to RAI dashboard and multilabel text classification covid events dataset by @imatiach-msft in microsoft#1811
      • Enable dropped_features for error analysis, data explorer and explainer, counterfactual and causal managers by @tongyu-microsoft in microsoft#1824

    • Error Analysis

      • Always display tree branching decisions by @tongyu-microsoft in microsoft#1870
      • Show branching decisions on hover over on unselected tree nodes by @tongyu-microsoft in microsoft#1881

    • RAI test utils

      • Add rai_test_utils package by @gaugup in microsoft#1777
      • Add release-rai-test-utils.yml to release rai_test_utils package by @gaugup in microsoft#1797
  • bug fixes and tests

    • Responsible AI Dashboard

      • Update e2e visit() function to not refresh the page if provided name doesn't exist in RAINotebookNames by @tongyu-microsoft in microsoft#1792
      • Fix TypeError: Cannot read properties of undefined (reading 'sort') by @vinuthakaranth in microsoft#1798
      • Fix multiple Scroll bars appear on Model demo page after setting the page to 320*256px viewport by @vinuthakaranth in microsoft#1796
      • Fix ‘Close button’ and ‘Spin button’ present in new cohort dialog does not have any title or aria-label by @vinuthakaranth in microsoft#1806
      • Fix fast pass accessibility bugs by @vinuthakaranth in microsoft#1817
      • Add min-width for highchart and fix padding by @vinuthakaranth in microsoft#1818
      • Add user feature name to box plot hover over by @gaugup in microsoft#1819
      • update to latest 16.x node version for tests by @imatiach-msft in microsoft#1809
      • Simplify route generation logic for testing with flights by @romanlutz in microsoft#1530
      • Use "lower whisker" and "upper whisker" instead of misleading "minimum" and "maximum" on boxplot tooltips by @romanlutz in microsoft#1825
      • Remove unused flight "newSdkEndpoints" by @gaugup in microsoft#1831
      • Gate call to probability distribution end point by @gaugup in microsoft#1839
      • Disable partial dependence plots for large data scenarios by @gaugup in microsoft#1843
      • add multilabel text example and fix correct and incorrect groups in data analysis by @imatiach-msft in microsoft#1848
      • Add number of cohort samples inside generateMetrics() by @gaugup in microsoft#1849
      • add multilabel RAI vision example for fridge multilabel dataset by @imatiach-msft in microsoft#1844
      • update raiwidgets to rai-core-flask v0.5.0 by @imatiach-msft in microsoft#1851
      • Remove warning for 'dropped_features are not in use currently.' by @tongyu-microsoft in microsoft#1871
      • add multilabel model overview metrics by @imatiach-msft in microsoft#1882
      • Populate probability box chart data on mount by @romanlutz in microsoft#1883
      • Improve spacing in RAI dashboard by @romanlutz in microsoft#1875
      • release raiwidgets and responsibleai 0.24.0 by @imatiach-msft in microsoft#1891
      • Refactor metric computation in ModelOverview by @gaugup in microsoft#1850
      • Remove setState call from render function in ProbabilityDistributionChart.tsx by @gaugup in microsoft#1858
      • Forecasting: boilerplate only by @romanlutz in microsoft#1887

    • RAIInsights

      • Update requirements.txt to pin econml to 0.14.0 by @gaugup in microsoft#1813
      • Remove pip pinning to unblock RAI gates by @gaugup in microsoft#1826
      • Update requirements-linting.txt to pin isort by @gaugup in microsoft#1847
      • fix loading older versions of RAIInsights due to _FEATURE_METADATA key error by @imatiach-msft in microsoft#1857
      • Add better error message if all features are dropped from dataset by @tongyu-microsoft in microsoft#1867
      • Refer to prediction wrappers from ml-wrappers by @gaugup in microsoft#1830
      • Raise exception instead of logging warnings when model cannot be loaded by @gaugup in microsoft#1886

    • Counterfactual

      • [Metadata] Fix counterfactual error on unable to update feature value by @tongyu-microsoft in microsoft#1842
      • Add test_data to ICounterfactualData by @gaugup in microsoft#1892
      • Add test_data to CounterfactualData for large data scenario by @gaugup in microsoft#1885
      • [Big data] Counterfactuals - part 1 by @vinuthakaranth in microsoft#1888
      • Add missing property 'id' in ICounterfactualData by @gaugup in microsoft#1834

    • Causal

      • Add e2e for dependency plot Avg of abs value option by @vinuthakaranth in microsoft#1768
      • Remove redundant call to serialize_json_safe() in raiwidgets causal tests by @gaugup in microsoft#1840
      • Fetching global causal effects and global policy from SDK backend by @gaugup in microsoft#1753
      • Remove typo casual and correcting it to causal by @gaugup in microsoft#1846
      • Disable local causal chart for large data support by @gaugup in microsoft#1845
      • Add better error message if causal treatment features contain dropped features by @tongyu-microsoft in microsoft#1868

    • Error Analysis

      • Update erroranalysis tests to use new rai_test_utils package by @gaugup in microsoft#1800
      • Fix Heatmap tooltip theme by @vinuthakaranth in microsoft#1808
      • impute missing values prior to calculating mutual information in erroranalysis by @imatiach-msft in microsoft#1807
      • update raiwidgets and responsibleai to erroranalysis 0.3.13 by @imatiach-msft in microsoft#1822
      • Migrate debug_ml and matrix endpoints to compute_error_tree_on_dataset() and compute_matrix_on_dataset() by @gaugup in microsoft#1795
      • in error analysis, refactor error rate metric into function by @imatiach-msft in microsoft#1742
      • fix error in erroranalysis heatmap caused by max value over right side of max bin by @imatiach-msft in microsoft#1866

    • Dataset Explorer

      • Add dataset analysis chart view for large data by @gaugup in microsoft#1829

    • Model Overview

      • Fix Model overview - Dataset cohorts]: At 320*256 viewport content is getting truncated and graph is not visible. by @vinuthakaranth in microsoft#1801
      • Model overview: add box plot in the regression case by @romanlutz in microsoft#1532
      • Send the correct column name to SDK backend for model overview distribution charts by @gaugup in microsoft#1835
      • Derive fairness metrics from metric stats instead of cohort stats by @gaugup in microsoft#1854
      • Add common APIs for model overview metrics by @gaugup in microsoft#1862

    • Interpret

      • Get global explanations from SDK if endpoint available by @gaugup in microsoft#1820
      • Disabling local feature importance view for large data scenario by @gaugup in microsoft#1828
      • Remove dead space in aggregate feature importance by @tongyu-microsoft in microsoft#1869
      • update responsibleai to interpret-community 0.28.0 by @imatiach-msft in microsoft#1879

    • NLP Feature Extractors

      • remove the unnecessary dependency pins in nlp-feature-extractors package by @imatiach-msft in microsoft#1759
      • release nlp-feature-extractors 0.1.0 by @imatiach-msft in microsoft#1821
      • remove unused dependency and method in nlp-feature-extractors package by @imatiach-msft in microsoft#1823

    • RAI Utils

      • Port cohort.py from raiwidgets to raiutils by @gaugup in microsoft#1859
      • Update raiutils to 0.3.0 in raiwidgets, responsibleai, erroranalysis and nlp_feature_extractors by @gaugup in microsoft#1873

  • other

    • Add Ke as code owner to remaining folders by @vinuthakaranth in microsoft#1799
    • Update CODEOWNERS by @tongyu-microsoft in microsoft#1805
    • merge postga branch into main by @gaugup in microsoft#1815
    • Remove postga build trigger by @gaugup in microsoft#1816
    • upgrade setup-python github action from v2 to v4 and pin to older ubuntu image for python 3.6 to fix build error by @imatiach-msft in microsoft#1838
    • Revert "Update some sample notebooks to include dropped_features" by @tongyu-microsoft in microsoft#1837
    • fix install command for speech to text example by @imatiach-msft in microsoft#1856
    • update rai-core-flask by closing WSGIServer on separate thread to fix segfault errors and get the rai-core-flask release builds working again by @imatiach-msft in microsoft#1852
    • Update maintainers by @romanlutz in microsoft#1860
    • Update requirements.txt in raiwidgets and responsibleai to pin numpy upper bound by @gaugup in microsoft#1864
    • Update requirements-linting.txt to remove pin on isort by @gaugup in microsoft#1861
    • Turn off fail-fast for python tests by @romanlutz in microsoft#1878
    • Separate out rai_core_flask CI and add paths filter by @romanlutz in microsoft#1884
    • Remove unused definitions, commented out code, etc. by @romanlutz in microsoft#1876
    • Update CI-raiwidgets-pytest.yml to skip windows 3.8 tests by @gaugup in microsoft#1890
    • Exempt only files from max-lines rule that are already in violation, but not new files by @romanlutz in microsoft#1896
    • skip CI-Python gates when pull requests are unrelated by @romanlutz in microsoft#1895

v0.23.0

  • educational materials
  • bug fixes and tests

    • Responsible AI Dashboard

      • Update UT - use new ErrorCohort() to mock data by @tongyu-microsoft in microsoft#1711
      • [A11y] 1.Error analysis tree node text alignment 2. Add ariaLabel for new cohort panel 3. Model overview chart fix by @RubyZ10 in microsoft#1719
      • reduce file size for e2e by @xuke444 in microsoft#1723
      • fix max line line error for vision by @xuke444 in microsoft#1725
      • clean up fairness by @xuke444 in microsoft#1724
      • rollback local importance selection by @xuke444 in microsoft#1726
      • add fridge dataset to test app for RAI vision dashboard and fix bug with large images in flyout by @imatiach-msft in microsoft#1730
      • fix images in data characteristics panel hiding the success/failure instances by @imatiach-msft in microsoft#1731
      • add test watch mode by @xuke444 in microsoft#1728
      • refactor vision dashboard to remove getDerivedStateFromProps by @imatiach-msft in microsoft#1732
      • [A11y]Show Error analysis tree focus && Add Screen reader for 1. New cohort panel, 2. Add component callout 3. Callout tooltip by @RubyZ10 in microsoft#1727
      • [A11y] Fix bug - Screen reader is not announcing the associated information with axis selection radio buttons in data analysis chart view by @RubyZ10 in microsoft#1736
      • Fix cohort deletion not working by @vinuthakaranth in microsoft#1738
      • fix e2e watch mode by @xuke444 in microsoft#1740
      • fix an reference issue for chart lib by @xuke444 in microsoft#1741
      • fix chart that generates duplicate plotly data by @xuke444 in microsoft#1744
      • [A11y] Fix table view keyboard access by @RubyZ10 in microsoft#1739
      • add data characteristics search and fix switching between tabs in RAI vision dashboard by @imatiach-msft in microsoft#1745
      • fix RAI dashboard for vision models crashing when selecting individual datapoints in data analysis by @imatiach-msft in microsoft#1733
      • remove duplicate settings button in RAI dashboard for images by @imatiach-msft in microsoft#1747
      • in data characteristics of RAI vision dashboard, fix bug where labels to display is reset when changing number of rows by @imatiach-msft in microsoft#1748
      • [A11y][screen reader] Announce component added after adding a component by @RubyZ10 in microsoft#1751
      • [A11y] 1. Hide chart heading & 2. Fix Add component callout high contrast focus indicator by @RubyZ10 in microsoft#1758
      • Fix full name not visible for features in highcharts by @vinuthakaranth in microsoft#1761
      • Filter out undefined filters to resolve a robot page exception by @tongyu-microsoft in microsoft#1764

    • RAIInsights

      • Correct predict call to predict_proba by @gaugup in microsoft#1717
      • Port over cohort filtering tests for 'Predicted Y' cohort to responsibleai by @gaugup in microsoft#1709
      • Raise exception in case categorical feature list doesn't have all string features by @gaugup in microsoft#1720
      • Pass classes into the sklearn confusion_matrix by @gaugup in microsoft#1674
      • fix flaky test test_rai_insights_add_save_load_save on dataset fetch by @imatiach-msft in microsoft#1734
      • Add validations to catch empty train/test scenarios by @gaugup in microsoft#1749
      • responsibleai: Move tests into respective directories by @gaugup in microsoft#1746

    • Counterfactual

      • Update requirements.txt to upgrade dice-ml to 0.9 by @gaugup in microsoft#1782

    • Causal

      • Add attributes 'outcome', 'feature' and 'feature_value' to causal global effects by @gaugup in microsoft#1750
      • Move causal only function from core-ui to causality by @gaugup in microsoft#1752

    • Error Analysis

      • Change multiclass F1_Score metrics to Macro_F1_Score/Micro_F1_Score by @gaugup in microsoft#1710
      • Fix "Clear” button of Search field under 'Feature List' dialog is not Functional and Focus indicator is not visible on all four sides for “Tree map” and "heat map" tabs. by @vinuthakaranth in microsoft#1718
      • Rename 'Error' filter name to 'Regression error' by @gaugup in microsoft#1729
      • [A11y] announce the displayed search results information of Feature List "Search" edit field. by @RubyZ10 in microsoft#1757
      • fix error analysis cohort filter for string label values by @imatiach-msft in microsoft#1766
      • fix isnan exception in erroranalysis for newer versions of pandas and numpy when calling qcut by @imatiach-msft in microsoft#1784
      • update raiwidgets and responsibleai to erroranalysis 0.3.11 by @imatiach-msft in microsoft#1767

    • Dataset Explorer

    • Model Overview

      • Fix [Keyboard navigation - Azure Machine Learning - Model overview - Dataset cohorts]: Focus indicator is overlapping with the "help me choose metrics" button. by @vinuthakaranth in microsoft#1716

    • Interpret

      • in RAI text dashboard, fix topK and maxK on slidebar not updating when selecting different text documents by @imatiach-msft in microsoft#1737
      • fix table view for image and text based on discussions for local importances and data analysis by @imatiach-msft in microsoft#1735
      • Fix dependency plot showing Average of abs value by @vinuthakaranth in microsoft#1765

    • NLP Feature Extractors

      • Bump protobuf from 3.17.3 to 3.18.3 in /nlp_feature_extractors by @dependabot in microsoft#1743

  • other

v0.22.0

  • educational materials
    • Notebook change showing the usage of identity feature and feature metadata by @gaugup in microsoft#1627
  • new features

    • Responsible AI Dashboard

      • Vision Dashboard Table View and initial explanation flyout by @jamesbchao in microsoft#1621
      • Vision dashboard backend without data characteristics tab by @jamesbchao in microsoft#1668
      • Vision dashboard data characteristics tab by @jamesbchao in microsoft#1675
      • Vision dashboard cohorts by @jamesbchao in microsoft#1686
      • Vision model overview metrics by @jamesbchao in microsoft#1688

    • Error Analysis

    • NLP Feature Extractors

      • Add the code for the nlp_feature_extractors to the RAI Toolbox by @ilmarinen in microsoft#1580
  • bug fixes and tests

    • Responsible AI Dashboard

      • [Accessibility] Data analysis scatter chart - use symbol and color to differentiate data series by @RubyZ10 in microsoft#1667
      • Bump moment-timezone from 0.5.34 to 0.5.35 by @dependabot in microsoft#1677
      • Add validations for dashboard endpoints by @gaugup in microsoft#1672
      • [Accessibility][reflow] Error analysis tree map & Data explorer & Feature importance Reflow by @RubyZ10 in microsoft#1678
      • [Accessibility][reflow] Error Analysis heat map & Model overview & Counterfactual & Causal analysis by @RubyZ10 in microsoft#1680
      • [localization] Localize high chart context menu items by @RubyZ10 in microsoft#1691
      • for RAI text dashboard, remove radio buttons and legend text when avg of abs value local importances selected by @imatiach-msft in microsoft#1683
      • Vision dashboard UI Fixups by @jamesbchao in microsoft#1687
      • touch ups from designer feedback to RAI dashboard for text data by @imatiach-msft in microsoft#1695

    • RAIInsights

      • Update econml to 0.13.1 by @gaugup in microsoft#1660
      • [responsibleai]: Add tests for classification outcome (multiclass) and regression error by @gaugup in microsoft#1662
      • JSON serialize the endpoint output and add tests by @gaugup in microsoft#1679
      • Cache predict() and predict_proba() output by @gaugup in microsoft#1702
      • Reduce the train and test dataset sizes in responsibleai to reduce test time by @gaugup in microsoft#1630

    • Counterfactual

    • Causal

      • Add validations for heterogeneity_features in causal manager by @gaugup in microsoft#1659
      • Fix localization for causal tooltips by @gaugup in microsoft#1690

    • Error Analysis

      • [Accessibility] Error analysis screen reader by @RubyZ10 in microsoft#1663
      • Add metric computation of mean prediction by @gaugup in microsoft#1693
      • Add support for 'True Y' filter for cohort filtering by @gaugup in microsoft#1685
      • Avoid additional predict call in case of ModelAnalyzer by @gaugup in microsoft#1701
      • release erroranalysis v0.3.10 by @imatiach-msft in microsoft#1706

    • Dataset Explorer

      • refactor table view into data analysis component by @imatiach-msft in microsoft#1673
      • add mini-table to feature importances view by @imatiach-msft in microsoft#1682

    • Model Overview

      • [Model Overview] Box plot data for classification probabilities from SDK endpoint by @tongyu-microsoft in microsoft#1689
      • Change 'Accuracy' to 'Accuracy score' in ModelOverview by @gaugup in microsoft#1700

    • Interpret

      • [Accessibility][reflow] Individual conditional expectation plot by @RubyZ10 in microsoft#1699

    • NLP Feature Extractors

      • update nlp-feature-extractors readme, dependencies and add a release script by @imatiach-msft in microsoft#1666
      • fix nlp-feature-extractors release by renaming requirements file by @imatiach-msft in microsoft#1670
      • add missing positive negative words csv file to attribute extractors by @imatiach-msft in microsoft#1671

  • other

    • [localization] LEGO: Pull request from JUNO/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20220907180147477 to main by @csigs in microsoft#1694
    • Update CODEOWNERS by @gaugup in microsoft#1696

v0.21.0

  • educational materials
    • [docs] Add Data Balance README by @ms-kashyap in microsoft#1567
    • Replace SynapseML links with RAIToolbox links by @ms-kashyap in microsoft#1592
  • new features

    • Responsible AI Dashboard

      • Data Balance UX Part 2: Add visualizations as a flight by @ms-kashyap in microsoft#1512
      • add new interpret-vision library skeleton by @jamesbchao in microsoft#1586
      • add support for multiclass local importance values for text by @imatiach-msft in microsoft#1601
      • Vision explanation dashboard -- Image explorer view by @jamesbchao in microsoft#1587

    • Error Analysis

      • Make generic class interface for cohort based filtering by @gaugup in microsoft#1597
  • bug fixes and tests

    • Responsible AI Dashboard

      • update release workflows to include interpret-text npm package by @imatiach-msft in microsoft#1582
      • Add RAI prefix for OSS UI telemetry events by @tongyu-microsoft in microsoft#1584
      • Bump moment from 2.29.2 to 2.29.4 by @dependabot in microsoft#1538
      • Bump eventsource from 1.0.7 to 1.1.1 by @dependabot in microsoft#1456
      • [Bugfix] Fix Dropdown bug and Update styling for feature balance measures by @ms-kashyap in microsoft#1583
      • Refactor tests in test_responsibleai_dashboard_input.py to have common success and failure criteria by @gaugup in microsoft#1591
      • update raiwidgets and responsibleai to erroranalysis 0.3.6 by @imatiach-msft in microsoft#1588
      • Fix tooltip location showing up incorrectly due to conflicting ids by @imatiach-msft in microsoft#1599
      • Update e2e test to expand correct predictions caret symbol by @vinuthakaranth in microsoft#1603
      • Updates to RAI text dashboard - update test app values to latest shap release, fix predicted label, limit selection to one row, fix random crash by @imatiach-msft in microsoft#1605
      • [300 line limit] Refactor AxisConfigDialog: create AxisConfigBinOptions and AxisConfigChoiceGroup component by @tongyu-microsoft in microsoft#1609
      • [300 line limit] Refactor CohortEditor, OverallMetricChart, set max-lines rule for core-ui package by @tongyu-microsoft in microsoft#1612
      • design updates for the RAI text dashboard local importances view by @imatiach-msft in microsoft#1611
      • [Accessibility] Fix all existing accessibility bugs by @RubyZ10 in microsoft#1613
      • Move CalculateBoxPlot.test.tsx to core-ui by @gaugup in microsoft#1620
      • Add feature metadata to IDataset and JointDataset by @gaugup in microsoft#1614
      • Remove console.log() statement from CohortEditor.tsx by @gaugup in microsoft#1628
      • update raiwidgets and responsibleai to erroranalysis 0.3.8 by @imatiach-msft in microsoft#1629
      • add new interpret-vision package to npm publish workflows in build and release gates by @imatiach-msft in microsoft#1615
      • [Accessibility] fix several accessibility bugs by @RubyZ10 in microsoft#1632
      • add UI tests for RAI text dashboard by @imatiach-msft in microsoft#1636
      • Fix typo for indices by @gaugup in microsoft#1640
      • Add extra space between Identity feature and the actual feature name by @gaugup in microsoft#1641
      • Use JointDataset API in ProcessPreBuiltCohort.ts by @gaugup in microsoft#1635
      • add wrapping for long text documents to text highlighting component in RAI dashboard for text by @imatiach-msft in microsoft#1643
      • add tests to RAI text dashboard importances part 2 by @imatiach-msft in microsoft#1642
      • add radio button tests for RAI text dashboard by @imatiach-msft in microsoft#1648
      • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 69976515 by @tongyu-microsoft in microsoft#1653
      • Add unit test for calculateLineData by @gaugup in microsoft#1654
      • Add test for uncovered condition in responsibleai_dashboard_input.py by @gaugup in microsoft#1638
      • update raiwidgets and responsibleai to erroranalysis 0.3.9 by @imatiach-msft in microsoft#1657

    • RAIInsights

      • responsibleai: Add FeatureMetadata class and related tests by @gaugup in microsoft#1604
      • Add categorical statistics computation in RAIInsights by @gaugup in microsoft#1598
      • Add cohort filtering capability in RAIInsights class by @gaugup in microsoft#1618
      • Inject FeatureMetadata into RAIInsights by @gaugup in microsoft#1616
      • responsibleai: Remove old manager namespaces by @gaugup in microsoft#1564
      • Fix docs in rai_base_insights.py by @gaugup in microsoft#1650
      • fix failing RAI Insights test due to change in EA _string_ind_data type by @imatiach-msft in microsoft#1658
      • Enable validations for FeatureMetadata when passed into RAIInsights by @gaugup in microsoft#1651

    • Counterfactual

      • [300 line limit] Refactor CounterfactualList by @tongyu-microsoft in microsoft#1619
      • Reduce the time taken by RAIInsights counterfactual tests by @gaugup in microsoft#1622
      • [300 line limit] Refactor CounterfactualChart by @tongyu-microsoft in microsoft#1624
      • Add identity feature value to counterfactual hover over by @gaugup in microsoft#1626
      • Minor refactoring in counterfactual package by @gaugup in microsoft#1634
      • [Accessibility] Add headings & change counterfactual text color by @RubyZ10 in microsoft#1646

    • Causal

      • Add backend functions in causal_manager.py to handle dashboard requests by @gaugup in microsoft#1579
      • SDK change for computing aggregate causal effects and policy tree per cohort by @gaugup in microsoft#1552
      • Add identity feature value to causal hover over by @gaugup in microsoft#1633

    • Error Analysis

      • Refactor categorical statistics computation in separate module by @gaugup in microsoft#1600
      • Add more classification metrics to erroranalysis by @gaugup in microsoft#1575
      • Improve error message when categorical string column is treated as numeric and error analysis is run by @imatiach-msft in microsoft#1562
      • fix bug in treeview filtering logic in error analysis package by @imatiach-msft in microsoft#1593
      • erroranalysis: Bug fix in classification outcome cohort filter by @gaugup in microsoft#1596
      • release erroranalysis 0.3.7 by @imatiach-msft in microsoft#1606
      • fix error analysis predictions analyzer failing due to new filtering refactor by @imatiach-msft in microsoft#1625
      • Make return type of string_ind_data in process_categoricals consistent by @gaugup in microsoft#1617
      • Handle 'Classification outcome' cohort filter for multiclass scenario in SDK by @gaugup in microsoft#1623
      • Handle 'Regression Error' cohort filter for regression scenario in SDK by @gaugup in microsoft#1644
      • [Accessibility] Error analysis tree node change 1. Bubble error rate colors; 2. Bubble text & fill by @RubyZ10 in microsoft#1649
      • release erroranalysis 0.3.9 by @imatiach-msft in microsoft#1655

    • Dataset Explorer

      • Add identity feature value to dataset explorer scatter plot by @gaugup in microsoft#1645

    • Model Overview

      • If feature groups is more than 10, short it to 10 with "Other" for the rest by @RubyZ10 in microsoft#1602
      • [Model Overview]Fix bug: x axis selection is not kept between tab switch by @RubyZ10 in microsoft#1607
      • [ModelOverview] Fix bug- On choosing a specific feature cohort and trying to choose feature cohort again - dropdown has all cohorts selected by @RubyZ10 in microsoft#1608
      • [ModelOverview]Fix bug- Model Overview: Clarification - Should state of 'use line chart' be persisted when switched from Probability -> Metrics -> Prob tab by @RubyZ10 in microsoft#1610
      • enable new Model Overview experience in OSS RAI Dashboard by default by @imatiach-msft in microsoft#1647

    • Interpret

      • Add backend functions in explainer_manager.py to handle dashboard requests by @gaugup in microsoft#1585
      • Add identity feature value to feature importance hover over by @gaugup in microsoft#1637

  • other

    • update CHANGES.md release notes to reflect delay in package release by @imatiach-msft in microsoft#1581
    • Remove strict pinnings from rai-core-flask requirements.txt. by @janjagusch in microsoft#1578
    • release rai-core-flask 0.5.0 by @imatiach-msft in microsoft#1589
    • update deprecated pypi publish github action to new supported version by @imatiach-msft in microsoft#1595
    • Update CODEOWNERS to add default reviewers for core-ui by @gaugup in microsoft#1639

v0.20.0

  • new features

    • Responsible AI Dashboard

      • Model overview: add legend to probability line chart by @romanlutz in microsoft#1492
      • Model overview: add cohort selection hint by @romanlutz in microsoft#1496
      • Model overview: wrap text after max line width by @romanlutz in microsoft#1503
      • Model overview: use spline instead of line chart by @romanlutz in microsoft#1508
      • Model overview: turn on heatmap colors by default by @romanlutz in microsoft#1510
      • Model overview: disable confirm button when no changes were made by @romanlutz in microsoft#1509
      • Add e2e tests for new model overview experience (part 1) by @romanlutz in microsoft#1467
      • Model overview: use unique key to avoid heatmap color mismatch by @romanlutz in microsoft#1517
      • Model overview: new cohorts should show up by @romanlutz in microsoft#1519
      • Model overview: bold cohort names in tooltip by @romanlutz in microsoft#1531
      • add text interpretability dashboard UI by @imatiach-msft in microsoft#1159
      • Data Balance UX Part 1: add data_balance_measures to IDataset by @ms-kashyap in microsoft#1511
      • refactor individual importances view to integrate local importances view for text data by @imatiach-msft in microsoft#1569

    • RAIInsights

      • Create DataBalanceManager + unit tests; clean up data balance conftest by @ms-kashyap in microsoft#1485
  • bug fixes and tests

    • Responsible AI Dashboard

      • [office-ui upgrade] Move components in causality, counterfactuals and dataset-explorer package to fluentui by @vinuthakaranth in microsoft#1481
      • [office-ui upgrade] Move components in error-analysis package to fluentui by @vinuthakaranth in microsoft#1484
      • [office-ui upgrade] Move components in interpret package to fluentui by @vinuthakaranth in microsoft#1495
      • [office-ui upgrade] Move components in fairness package to fluentui by @vinuthakaranth in microsoft#1494
      • [office-ui upgrade] Move components in model-assessment & dependencies to fluentui by @vinuthakaranth in microsoft#1497
      • [office-ui upgrade] Move ChoiceGroup & all other remaining dependencies to fluentui by @vinuthakaranth in microsoft#1501
      • [Bugfix] HeatmapHighChart: Prioritize configOverride over defaultOptions by @ms-kashyap in microsoft#1490
      • Remove download svg option due to highcharts limitations by @tongyu-microsoft in microsoft#1498
      • Upgrade highcharts from v9.3.0 to v10.1.0, remove highcharts bundle by setting it to external module by @tongyu-microsoft in microsoft#1506
      • Fix highcharts tooltip is clipped by @tongyu-microsoft in microsoft#1502
      • Bump shell-quote from 1.7.2 to 1.7.3 by @dependabot in microsoft#1514
      • [UI refactor] Remove all alert functions and set no-alert rule by @tongyu-microsoft in microsoft#1518
      • Use of "==" and "!=" confusing to non-programmers, update operators to strings by @tongyu-microsoft in microsoft#1525
      • [UI refactor] Remove .bind() part 1 - Remove .bind(this, true/false) by @tongyu-microsoft in microsoft#1523
      • Panel buttons should be uniform: change all buttons to "Apply"; all buttons should be at bottom by @tongyu-microsoft in microsoft#1524
      • Fix slider bug by @vinuthakaranth in microsoft#1526
      • Fix e2e-widget.js for the case without flight argument by @romanlutz in microsoft#1529
      • Make the local point selection drop down consistent in causal and counterfactual by @gaugup in microsoft#1539
      • [UI refactor] Remove .bind() part2 - Remove .bind in counterfactual folder and part of .bind() in other folders by @tongyu-microsoft in microsoft#1542
      • [UI refactor] Remove .bind() part3 - Remove .bind for SpinButton in AxisConfigDialog, MultiICEPlot and Fairness by @tongyu-microsoft in microsoft#1555
      • refactor text highlighting component to use Text and Label from FluentUI instead of styles on spans by @imatiach-msft in microsoft#1558
      • centralize plot colors to core ui by @imatiach-msft in microsoft#1559
      • create other cohort in model overview when there are more than 10 cohorts by @imatiach-msft in microsoft#1572
      • [Telemetry] Add button click event tracking for main menu and error analysis component by @tongyu-microsoft in microsoft#1570
      • Remove redundant lines of code in StatisticsUtils.ts by @gaugup in microsoft#1571
      • [Telemetry] Add button click event trackers for new model overview, data explorer, feature importances, counterfactual and causal by @tongyu-microsoft in microsoft#1574

    • RAIInsights

      • update responsibleai package to latest interpret-community 0.26.0 by @imatiach-msft in microsoft#1491
      • Update requirements.txt to bump up econml to 0.13.0 by @gaugup in microsoft#1499
      • remove version for flask by @xuke444 in microsoft#1515
      • Remove unused variables from create_classification_pipeline() by @gaugup in microsoft#1520
      • update raiwidgets and responsibleai to latest erroranalysis 0.3.3 release by @imatiach-msft in microsoft#1535
      • Add column and feature range meta data to meta file by @vinuthakaranth in microsoft#1541
      • update raiwidgets and responsibleai to erroranalysis 0.3.4 by @imatiach-msft in microsoft#1544
      • update packages to new raiutils 0.2.0 and rai-core-flask 0.4.0 releases by @imatiach-msft in microsoft#1550
      • Add hierarchy to rai exceptions by @gaugup in microsoft#1568
      • Add multiclass classification test in raiwidgets by @gaugup in microsoft#1576

    • Counterfactual

      • Update counterfactual serialization via dice-ml APIs by @gaugup in microsoft#1479
      • Fix unable to input decimal point in what if counterfactual feature text field by @tongyu-microsoft in microsoft#1459
      • Bold input text for values that have changed from the original reference values for counterfactual list by @tongyu-microsoft in microsoft#1516
      • Fix bug in deserializing counterfactual explainer by @gaugup in microsoft#1554
      • Add ID to counterfactual config and serialized counterfactual data by @gaugup in microsoft#1553

    • Causal

    • Error Analysis

      • add more docs to matrix filter aka heatmap in error analysis by @imatiach-msft in microsoft#1489
      • part 2: add pyspark support to tree surrogate model in error analysis by @imatiach-msft in microsoft#1388
      • use new getNativeModel method in get_surrogate_booster_pyspark to simplify erroranalysis code by @imatiach-msft in microsoft#1475
      • fix erroranalysis erroring on serialization with numpy int64 feature names by @imatiach-msft in microsoft#1537

    • Dataset Explorer

      • Reorder controls in data explorer legend by @tongyu-microsoft in microsoft#1493
      • Fill color for data explorer box plot by @tongyu-microsoft in microsoft#1504

  • other

    • Add issues template for bugs and feature requests by @imatiach-msft in microsoft#1486
    • Taking dependency on is_classifier() method and SKLearn constants class from raiutils by @gaugup in microsoft#1289
    • Localized file check-in by OneLocBuild Task: Build definition ID 21368: Build ID 65085801 by @riedgar-ms in microsoft#1500
    • Split raiwidgets/e2e-notebooks test pipeline in two by @romanlutz in microsoft#1478
    • Derandomize notebooks by @romanlutz in microsoft#1507
    • Unify naming between notebooks and test cases by @romanlutz in microsoft#1505
    • Update CODEOWNERS by @romanlutz in microsoft#1513
    • Add License and Requirements to Source Distribtion by @janjagusch in microsoft#1522
    • Move references for _convert_to_list() and _convert_to_string_list_dict() to raiutils by @gaugup in microsoft#1534
    • Move references for serialize_json_safe() to raiutils by @gaugup in microsoft#1536
    • fix flaky test by adding retry logic to fetch openml dataset in notebook by @imatiach-msft in microsoft#1547
    • release rai-core-flask 0.4.0 by @imatiach-msft in microsoft#1546
    • release raiutils v0.2.0 by @imatiach-msft in microsoft#1545
    • update sphinx dependencies to fix jinja2 doc build error by @imatiach-msft in microsoft#1551
    • Update CODEOWNERS by @tongyu-microsoft in microsoft#1557
    • [E2ETests]Add flight && screenshots by @RubyZ10 in microsoft#1566
    • fix release pipeline failing on UI widget tests by @imatiach-msft in microsoft#1573
    • fix e2e UI tests incorrectly running tests with flights when no notebook name specified and reduce number of test jobs by @imatiach-msft in microsoft#1577

v0.19.0

  • educational materials
    • add info about PR reviews to the contributing docs file by @imatiach-msft in microsoft#1472
    • Adding extra statement in PR template for e2e by @vinuthakaranth in microsoft#1461
  • new features

    • Responsible AI Dashboard

      • Model overview: improve placeholder, put confirm/cancel buttons at the bottom, and hide toggle when needed by @romanlutz in microsoft#1433

    • RAIInsights

      • Add DataBalanceAnalysis module to ResponsibleAI package to be consumed by visualizations by @ms-kashyap in microsoft#1473
  • bug fixes and tests

    • Responsible AI Dashboard

      • Fix locators logic for string features - data explorer component by @vinuthakaranth in microsoft#1441
      • Fix cohort name conflict and not run few tests for AML by @vinuthakaranth in microsoft#1442
      • Few e2e tests changes to accommodate AML static tests by @vinuthakaranth in microsoft#1445
      • Fix locators logic for string features - data explorer and model statistics components by @vinuthakaranth in microsoft#1446
      • Add more unittests RAI dashboard input class by @gaugup in microsoft#1448
      • Update the way to get the length of elements obtained in e2e tests by @vinuthakaranth in microsoft#1450
      • Fix dataset explorer plot with count not displaying by @imatiach-msft in microsoft#1454
      • Add toggle for switching classes in binary classification case by @imatiach-msft in microsoft#1444
      • [office-ui upgrade] Move checkbox to fluentui by @vinuthakaranth in microsoft#1465
      • [office-ui upgrade] Move combobox to fluentui by @vinuthakaranth in microsoft#1469
      • [office-ui upgrade] Move components in core-ui packageto fluentui by @vinuthakaranth in microsoft#1471
      • Fix TypeError: Cannot read properties of undefined (reading 'treatAsCategorical')" by @vinuthakaranth in microsoft#1452
      • Update predicted class value to uneditable text in counterfactual panel by @tongyu-microsoft in microsoft#1458
      • Collapse correct prediction by default for individual feature importance by @tongyu-microsoft in microsoft#1474

    • RAIInsights

      • Update requirements.txt to pin dice-ml at 0.8 by @gaugup in microsoft#1470

  • other

    • Add more utilities into raiutils by @gaugup in microsoft#1295
    • Release raiutils v0.1.0 by @imatiach-msft in microsoft#1480
    • Update responsibleai to raiutils 0.1.0, raiwidgets and responsibleai to erroranalysis 0.3.2 by @imatiach-msft in microsoft#1483
    • Merge postbuild branch back into main by @romanlutz in microsoft#1466
    • Remove postbuild branch triggers by @romanlutz in microsoft#1476

v0.18.2

  • bug fixes and tests

    • Responsible AI Dashboard

      • Bug fixes on 'Set value' not copying over feature values correctly in what if counterfactual panel by @tongyu-microsoft in microsoft#1416
      • Fix description for model overview by @romanlutz in microsoft#1425
      • Fix math.min / max for array size more than 10^7 by @xuke444 in microsoft#1427

    • RAIInsights

      • Add warning in counterfactual manager when unable to load explainer by @gaugup in microsoft#1412

    • Counterfactual

      • Remove "Set Value" blurb in case it is not available in counterfactual panel by @gaugup in microsoft#1426
      • Add y-axis description to counterfactual feature importance chart by @gaugup in microsoft#1423

    • Causal

      • Fix causal UI strings according to classification/regression tasks by @gaugup in microsoft#1419
      • Add the user class name to causal UI strings by @gaugup in microsoft#1422
      • Upper bound SciKit-Learn to address freeze in causal by @riedgar-ms in microsoft#1432

    • Error Analysis

      • Fix error on machines with pyspark installed where passed dataframe is not spark pandas by @imatiach-msft in microsoft#1415
      • Fix failing to create error report when filter_features is empty list by @imatiach-msft in microsoft#1421

    • Interpret

      • Filter out missing values from what if dropdown to prevent explanation dashboard from crashing by @imatiach-msft in microsoft#1418
      • Fix dependency chart axis updating with incorrect values in explanation dashboard by @imatiach-msft in microsoft#1437

  • other

    • Add postbuild branch trigger by @romanlutz in microsoft#1417
    • Upgrade numpy to fix random segfault test failures by @imatiach-msft in microsoft#1424
    • Fix flaky notebook causing build failures by adding retry logic by @imatiach-msft in microsoft#1431
    • Fix codecov and widget test screenshot uploads by @imatiach-msft in microsoft#1428

v0.18.1

  • educational materials
    • Add pre-built cohort into adult census notebook by @gaugup in microsoft#1243
    • Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb by @gaugup in microsoft#1252
  • new features

    • Responsible AI Dashboard

      • Add a summary for disaggregated analysis showing ratio and difference by @romanlutz in microsoft#1367
      • Use global cohort for disaggregated analysis by @romanlutz in microsoft#1370
      • Add accuracy as multiclass metric by @romanlutz in microsoft#1371
      • Add a top-level pivot to model overview by @romanlutz in microsoft#1375
      • Add a toggle to model overview to turn heatmap colors on and off by @romanlutz in microsoft#1376
      • Add feature and metric configuration flyouts to model overview by @romanlutz in microsoft#1381
      • Add probability distribution line chart to model overview by @romanlutz in microsoft#1382
      • Provide feature flight support to dashboard, test environment, and Python widget by @romanlutz in microsoft#1385

    • Error Analysis

      • add pyspark support to tree surrogate model in error analysis by @imatiach-msft in microsoft#1251
  • bug fixes and tests

    • Responsible AI Dashboard

      • Fix bug on scatter points in dataset explorer; remove outlier for aggregate feature importance box chart by @tongyu-microsoft in microsoft#1364
      • Change procedural loops to functions map() and filter() by @gaugup in microsoft#1255
      • Create rai-e2e package for tests shared with AML by @vinuthakaranth in microsoft#1378
      • Create e2e package for tests shared with AML by @vinuthakaranth in microsoft#1383
      • fix race condition in tree view update code which can cause root stats to be ignored on selected cohort in RAI dashboard static view by @imatiach-msft in microsoft#1386
      • String updates for model overview by @romanlutz in microsoft#1387
      • Some fixes for data explorer, heatmap, counterfactual components by @tongyu-microsoft in microsoft#1389
      • disable causal treatment whatif in static view by @imatiach-msft in microsoft#1391
      • Fix feature selection state carry-over bug in model overview by @romanlutz in microsoft#1390

    • RAIInsights

      • Rename _validate_model_analysis_input_parameters() to _validate_rai_insights_input_parameters() by @gaugup in microsoft#1373

    • Counterfactual

      • Add missing desired range to counterfactual chart by @gaugup in microsoft#1366
      • Cache and serialize the counterfactual explainer by @gaugup in microsoft#1374
      • fix big data set for counter factual chart by @xuke444 in microsoft#1393

    • Causal

      • Handle the loading of causal models more gracefully by @gaugup in microsoft#1377
      • Alignment and style fixes for causal by @tongyu-microsoft in microsoft#1392

  • other

v0.18.0

  • educational materials

    • Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb by @gaugup in microsoft#1195
    • Add supported models, data types and capability matrix to README.md responsibleai by @gaugup in microsoft#1259
    • make getting-started notebook a markdown file showing APIs by @imatiach-msft in microsoft#1223
    • fix readme link to fairness and interpretability example notebook by @imatiach-msft in microsoft#1282

  • new features

    • Responsible AI Dashboard

      • Replace dependence plot with highchart lib by @zhb000 in microsoft#1208
      • Add user defined cohort injection logic into raiwidgets by @gaugup in microsoft#1237
      • Add feature importance box & bar chart by @zhb000 in microsoft#1241
      • PreBuilt cohorts UX changes by @gaugup in microsoft#1242
      • Add individual causal scatter chart by @zhb000 in microsoft#1258
      • Add what-If scatter chart from highchart lib by @zhb000 in microsoft#1262
      • add new RAI Utils package for common utilities shared across RAI packages by @imatiach-msft in microsoft#1280
      • Add ICE chart by @zhb000 in microsoft#1283
      • Add highchart for Dataset Explorer by @zhb000 in microsoft#1286
      • Add disaggregated analysis table/heatmap by @romanlutz in microsoft#1332
      • Add disaggregated analysis table to Model Overview by @romanlutz in microsoft#1341

  • bug fixes and tests

    • Responsible AI Dashboard

      • update raiwidgets to rai-core-flask 0.2.5 release by @imatiach-msft in microsoft#1221
      • Add e2e tests for Housing decision making and multiclass dnn notebooks by @vinuthakaranth in microsoft#1212
      • Refactor highchart defaultOptions by @zhb000 in microsoft#1220
      • fix categorical what-if in RAI dashboard by @imatiach-msft in microsoft#1225
      • Add scatter highchart get coordinate for e2e by @zhb000 in microsoft#1226
      • update several required dependencies by @imatiach-msft in microsoft#1219
      • add ut for DashboardSettingDeleteButton by @xuke444 in microsoft#1231
      • Create pytest fixtures raiwidgets tests by @gaugup in microsoft#1232
      • Refactor dependence plot by @zhb000 in microsoft#1230
      • Add regression test for pre-defined cohorts in raiwidgets by @gaugup in microsoft#1249
      • Set bar color to align with plot style by @zhb000 in microsoft#1248
      • fix notebook build failures due to pywinpty dependency release failing in python 3.6 by @imatiach-msft in microsoft#1257
      • refactor tabs out of RAI dashboard into a separate component by @imatiach-msft in microsoft#1256
      • minor fix to url for responsibleai package in setup.py by @imatiach-msft in microsoft#1260
      • fix whitespace in values of adult census income dataset by @imatiach-msft in microsoft#1263
      • allow rai text insights to work with RAI dashboard by @imatiach-msft in microsoft#1269
      • remove duplicate code in explanation dashboard by @imatiach-msft in microsoft#1266
      • Allow duplicating cohorts multiple times by @romanlutz in microsoft#1274
      • Disable column header highlighting on hover in IndividualFeatureImportanceView by @romanlutz in microsoft#1272
      • Rename new cohorts from "Unsaved" to "Temporary cohort" by @romanlutz in microsoft#1273
      • Reorder "Sort by" controls in local feature importance chart by @zhb000 in microsoft#1281
      • Fix cohort info styling by @romanlutz in microsoft#1277
      • update docstring for explanation dashboard in regards to min number of rows by @imatiach-msft in microsoft#1271
      • make builds more reliable by adding retry logic to urlretrieve calls in notebooks by @imatiach-msft in microsoft#1218
      • upgrade pytest to 7.0.1, remove mock and upgrade pytest-mock to 3.6.1 by @imatiach-msft in microsoft#1287
      • fix jinja2 build error and remove deprecated codecov parameter by @imatiach-msft in microsoft#1293
      • Fix min/max special case in cohort filter creation with "in the range of" by @romanlutz in microsoft#1279
      • Rename 'Dashboard navigation' to 'Dashboard configuration' by @gaugup in microsoft#1291
      • Add raiutils to PR template by @gaugup in microsoft#1290
      • Fix heatmap bug by @zhb000 in microsoft#1297
      • Make "save and switch" work from cohort settings by @romanlutz in microsoft#1276
      • add retry logic to codecov step and only upload results for one python version and platform by @imatiach-msft in microsoft#1298
      • add github action to release raiutils to pypi by @imatiach-msft in microsoft#1294
      • Update requirements-linting.txt to add flake8-pytest-style by @gaugup in microsoft#1296
      • Fix sort by absolute value from local importance chart by @zhb000 in microsoft#1299
      • Rename "base cohort" to "global cohort" by @romanlutz in microsoft#1278
      • fix codecov comment not appearing on PRs by @imatiach-msft in microsoft#1302
      • take absolute value of error calculation for regression scenario by @imatiach-msft in microsoft#1301
      • Limit individual feature importance selection to up to 5 by @tongyu-microsoft in microsoft#1305
      • Add to_json() and from_json() methods to Cohort class by @gaugup in microsoft#1300
      • Add a highchart heatmap helper class by @romanlutz in microsoft#1307
      • Fix cohort setting string by @zhb000 in microsoft#1304
      • Fix Inconsistent font size on "All data" cohort by @zhb000 in microsoft#1303
      • Add a feature flag for the new model overview experience by @romanlutz in microsoft#1306
      • Clean up charts code by @romanlutz in microsoft#1313
      • Bump minimist from 1.2.5 to 1.2.6 by @dependabot in microsoft#1292
      • fix random node download failures by upgrading to latest github action with retry logic by @imatiach-msft in microsoft#1317
      • Add dataset cohort table to new ModelOverview experience by @romanlutz in microsoft#1314
      • Add installation instructions for raiwidgets to README by @imatiach-msft in microsoft#1320
      • refactor RAIInsights into RAIBaseInsights class for basic functionality by @imatiach-msft in microsoft#1284
      • Raise UserConfigValidationException in case no model but valid model serializer by @gaugup in microsoft#1325
      • show shift to an empty cohort in tree view as an empty node by @imatiach-msft in microsoft#1318
      • Bug fixing by @zhb000 in microsoft#1326
      • Add box outlier for dataset explorer by @zhb000 in microsoft#1323
      • Update string when no datapoint selected for local importance by @zhb000 in microsoft#1331
      • Fix Big empty space for featureImportance chart by @zhb000 in microsoft#1328
      • Change warning message to user exception for model type and task type mismatch by @gaugup in microsoft#1330
      • Limit each component description width up to 750px for readability by @tongyu-microsoft in microsoft#1336
      • block empty cohort creation in RAI Dashboard by @imatiach-msft in microsoft#1335
      • Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' by @tongyu-microsoft in microsoft#1339
      • All component title and descriptions should be aligned by @vinuthakaranth in microsoft#1346
      • Remove 5K limit blurb from local explanations tab by @gaugup in microsoft#1347
      • Bump moment from 2.28.0 to 2.29.2 by @dependabot in microsoft#1333 in microsoft#1350
      • disable turbo checking for large amount of data by @xuke444 in microsoft#1351
      • force re-render when chart type changes by @xuke444 in microsoft#1354
      • Show column chart for categorical feature in data explorer by @tongyu-microsoft in microsoft#1355
      • update fluentui by @xuke444 in microsoft#1356
      • update code owner by @xuke444 in microsoft#1308
      • update version to match studio by @xuke444 in microsoft#1357
      • A few UI alignment fixes by @tongyu-microsoft in microsoft#1359

    • RAIInsights

      • Add heterogeneity_model checks by @gaugup in microsoft#1210
      • DOC add type annotations to responsibleai package by @romanlutz in microsoft#1214
      • Add data validations to SDK defined cohorts by @gaugup in microsoft#1227
      • Pin markupsafe and itsdangerous to unblock gates by @gaugup in microsoft#1238
      • Make cohortData empty list in case no pre-defined cohorts are injected by @gaugup in microsoft#1247
      • Make _cohort.py module a public module by @gaugup in microsoft#1253

    • Counterfactual

      • Counterfactual Chart: Correct target description according to task_type by @gaugup in microsoft#1261
      • Counterfactual style refactor by @zhb000 in microsoft#1275
      • Add error message for counterfactual panel by @tongyu-microsoft in microsoft#1310
      • Change the counterfactual text color from black to grey by @gaugup in microsoft#1337
      • Expand the counterfactual flyout to cover the full page by @gaugup in microsoft#1315
      • Fix what if counterfactual header and description text misaligned by @zhb000 in microsoft#1316
      • Rename counterfactual style files to confirm with -.styles.ts by @gaugup in microsoft#1338
      • Sort features by default in counterfactual flyout by @gaugup in microsoft#1312
      • Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text by
      • Rename output column name in the counterfactual flyout by @gaugup in microsoft#1353

    • Error Analysis

      • fix tree api being called twice on initial load due to uninitialized context being used by @imatiach-msft in microsoft#1229
      • fix total metric changing with different num bins when using quantile binning on diabetes dataset by @imatiach-msft in microsoft#1233
      • erroranalysis version bump in raiwidgets to 0.1.31 by @imatiach-msft in microsoft#1245
      • add clear temporary cohort button to error analysis by @imatiach-msft in microsoft#1322
      • move the root all data statistics to ErrorReport and ErrorAnalysisData by @imatiach-msft in microsoft#1344
      • update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected by @imatiach-msft in microsoft#1349
      • Disable save as new cohort button if nothing is selected in error tree by @imatiach-msft in microsoft#1327
      • update erroranalysis to 0.2.1 by @imatiach-msft in microsoft#1334
      • move the root all data statistics to ErrorReport and ErrorAnalysisData (part 2) by @imatiach-msft in microsoft#1352

    • Interpret

      • update responsibleai to interpret-community 0.25.0 by @imatiach-msft in microsoft#1343

    • Causal

      • Individual causal style responsive by @zhb000 in microsoft#1268
      • Add test case for handling different types in causal manager by @gaugup in microsoft#1321
      • Rename causal style files to confirm with -.styles.ts by @gaugup in microsoft#1342

  • other

    • Remove widget tests from CI-notebook pipeline by @vinuthakaranth in microsoft#1213
    • add missing release steps causing rai-core-flask release errors by @imatiach-msft in microsoft#1216
    • fix release pipeline by adding pytorch packages for tests by @imatiach-msft in microsoft#1222
    • fix release error on unknown shell command when uploading to pypi by @imatiach-msft in microsoft#1224
    • Correct falsey to falsely by @gaugup in microsoft#1228
    • add a builddebug yarn command to build UX locally which can be debugged in browser e2e by @imatiach-msft in microsoft#1265

v0.17.0

  • educational materials
    • add e2e test steps to CONTRIBUTING.md file for developers/contributors
    • add steps for validating UX and SDK changes to documentation
    • replace boston housing dataset with california housing dataset in tests and example notebook due to ethical concerns and biases
    • update dnn pytorch notebook example to give more interesting results by increasing model epochs
  • new features

    • Responsible AI Dashboard

      • move performance metric chart, disaggregated metric table, and StatisticsUtils into core-ui
      • add highchart, all components will be moved to highchart instead of plotly gradually
      • add metrics to StatisticsUtils - F1 score, selection rate, mean absolute error
      • add cohort and filter definitions in raiwidgets SDK
      • enable highchart exporting functionality

    • Responsible AI Dashboard for Text

      • add new interpret-text library skeleton

    • Causal

      • add treatment bar chart
  • bug fixes and tests

    • Responsible AI Dashboard

      • add e2e UI tests for Causal Analysis - classification tests
      • add e2e UI tests for Feature Importance - aggregate importances
      • add e2e UI tests for Feature Importance - individual importances
      • add e2e UI tests for What-If Counterfactuals
      • add e2e UI tests for Data Explorer - diabetes-regression-model-debugging notebook
      • add e2e UI tests for Causal Analysis - diabetes-regression-model-debugging notebook
      • add e2e UI tests for Model Overview - diabetes-regression-model-debugging notebook
      • add e2e UI tests for responsibleaidashboard-housing-classification-model-debugging notebook
      • fix e2e UI What-if sort test
      • add e2e UI tests for Feature Importance - responsibleaidashboard-diabetes-decision-making notebook
      • add e2e UI tests for What-if, Model Overview and Data explorer - responsibleaidashboard-diabetes-decision-making notebook
      • add e2e UI tests for Aggregate Causal Analysis - diabetes-decision-making notebook
      • add multiclass classification dataset to UI test app and set up basic model-assessment tests for Model Overview
      • add notebook test for pytorch example notebook and add pytorch to CI
      • move to new fluentUI dependency
      • correctly annotate python warnings
      • separate out raiwidgets and UI tests into its own workflow, and added new pipeline to run RAIwidget e2e tests
      • add js and raiwidget code coverage
      • add upload step to e2e test for CI RAIWidgets Python Typescript pipeline
      • add upload step to e2e test to CI notebooks pipeline
      • return exception details to UI
      • add docstrings for locale parameter

    • RAIInsights

      • catch dirty data conditions in RAI Insights validation
      • update responsibleai to interpret-community 0.24.0
      • update responsibleai to interpret-community 0.24.1
      • update responsibleai to interpret-community 0.24.2
      • make responsibleai managers public and add back compat connectors
      • make explanations save behavior similar to other managers
      • change logic to walk categorical columns in validate_train_test_categories
      • cover missing test scenario in state_directory_management.py
      • default categorical feature to array
      • add error analysis test for regression task in RAIInsights
      • make causal manager add and compute behavior similar to other managers

    • Counterfactual

      • add feature importance description for counterfactual local importance
      • update Counterfactual explanation text description

    • Error Analysis

      • fix importances taking too long to calculate for large data in error analysis
      • add a perf test for validating training the surrogate error tree on large data and a minor optimization for traversing the tree
      • add filtering on index to error analysis backend for RAI dashboard cohort switch
      • remove unnecessary multiple calls to tree api
      • refactor tree view state to be private static in tree view renderer and not in parent
      • create github action to release erroranalysis
      • add excludes cohort filter test in erroranalysis
      • refactor matrix filter and area state to be private static
      • make search case insensitive in feature list
      • add more docs for error analysis surrogate tree
      • optimize tree traversal logic in error analysis to reduce execution time when running on several million rows

    • Interpret

      • allow ExplanationDashboard to show limited view for more than 1k features instead of erroring

    • Causal

      • improve documentation for nuisance_model and heterogeneity_model
      • implement list method in causal manager

  • other

    • fix pypi releases by reverting back to pypi token per package
    • fix dummy counterfactual feature importance values for UI test app
    • add pseudo language for localization test
    • integrate codecov into responsible-ai-toolbox repository
    • remove test warning due to use of deprecated np.object
    • add python 3.9 tests
    • fix ipython dependency in rai-core-flask and remove from raiwidgets
    • update setup.py files to carry python 3.9 classifiers
    • fix broken builds due to pip upgrade which broke pip-tools
    • remove mac temporarily from rai widgets step to fix failing builds
    • remove node-version from CI-python.yml
    • fix flaky flask test
    • improve description for responsibleai and raiwidgets pypi packages

v0.16.0

  • educational materials
    • major readme refactoring changes + filename changes for public release of RAI Dashboard
    • add missing doc for pred_y_dataset and min_child_samples
    • add npm and yarn installation instructions
  • new features

    • Counterfactual

      • counterfactual list style updates: highlight counterfactual examples columns with different color, and bold size
      • highlight first and last row of counterfactual list
      • implement list method for counterfactual manager

    • Error Analysis

      • add scrollbar area around matrix cells when there are many categories
      • add warning message for downsampled explanation data in ErrorAnalysis
  • bug fixes and tests

    • Responsible AI Dashboard

      • add e2e UI tests for Dataset Explorer - whisker plot
      • add e2e UI tests for Dataset Explorer - scatter plot
      • add e2e UI tests for Dataset Explorer - flyout
      • add e2e UI tests for Dataset Explorer - cohort scenarios
      • add e2e UI tests for Model Statistics - chart tests
      • refactor binding this with arrow func
      • add error message for duplicate cohort name
      • handle context cohort delete cases for data explorer

    • RAIInsights

      • throw error on mutating predict function in RAIInsights
      • use spilt orientation to save train and test dataframe to address duplicate indices condition in them
      • add .json extension to dtypes files and add tests
      • add predict and predict_proba outputs to RAIInsights save method
      • refactor RAIInsights load function into smaller functions
      • refactor classifier detection logic into common function
      • fix issue with missing directories
      • gracefully handle failure in loading model.pkl from disk
      • remove unused test dependencies responsibleai
      • add missing unit tests for _convert_to_list()
      • separate out save() and load() tests
      • add more unit tests for serialize_json_safe()
      • update responsibleai to interpret-community 0.23.0

    • Counterfactual

      • add e2e UI tests for what-if counterfactuals - creating counterfactuals

    • Error Analysis

      • fix coverage calculation for cell tooltips in heatmap for recall, precision and f1 score metrics
      • fix handling numpy types in heatmap for categories

    • Interpret

      • improve state management for explainer manager
      • test for save/load/save bug in Explanations
  • other
    • fix top broken image link in tour notebook on github
    • add isort to build pipelines
    • fix SubBarChart flaky test
    • update notebook tests for new notebooks
    • add flake8-bugbear for better python code contructs
    • add flake8-breakpoint to avoid code checkin with active breakpoints
    • remove csv files from source for cognitive services example notebooks
    • add yarn e2e-watch shortcut
    • replace existing user token with shared account token for releases

v0.15.1

  • educational materials
    • Notebooks updated for new class names (ResponsibleAIDashboard etc.)
  • new features
    • None
  • breaking changes

    • RAI Insights/Model Analysis

      • Rename explainers directory to generators when saving RAI Insights object
  • bug fixes

    • RAI Insights/Model Analysis

      • Fix Issue #1046, related to a save/load/save cycle when no explainer present
  • other
    • Extra tests for WhatIf counterfactuals
    • Checks on the UUID directory structure of RAI Insights

v0.15.0

  • new features

    • Model Analysis

      • move raw data into /data directory
      • perform schema validations during compute and load
      • rename model analysis to rai insights
      • rename ModelAnalysisDashboard to ResponsibleAIDashboard
      • directory based state management for model analysis tools
      • add .json extension to train and test dataset

    • Error Analysis

      • port ErrorReport schema validation for error analysis to responsibleai
      • add error analysis error importances (aka correlation of features with error) to model analyzer API
      • add support for special classification outcome filter to error analysis backend to work with model analysis filters

    • Counterfactual

      • remove save as new data point option for read-only mode on What-if counterfactual
      • split counterfactual output into individual json files
      • add support for non-string categorical values
  • bug fixes

    • Model Analysis

      • fix cohort text out of button box
      • fix bug when the tree node has 0 errors and N correct, the raw data table shows the opposite with N+1 errors and -1 correct.
      • when classes are not passed, sort them automatically in RAIInsights
      • fix composite filter issues

    • Counterfactual

      • fix sort feature columns by counterfactual

    • Error Analysis

      • fix map shift text not being clear, provide conditional dialog text based on current view
      • fix tree map connection line doubled after switching to dark theme

  • other

    • update readme links to be absolute instead of relative path
    • add more UX IFI tests for responsibleaitoolbox-classification-model-assessment notebook
    • several E2E UX test fixes
    • add E2E UX tests for Feature Importance - Cohort functionality
    • add test to render ModelAnalysisDashboard after save and load
    • add test to render ResponsibleAIDashboard after save and load
    • add E2E tests for What-if Counterfactuals - common functionalities & Y Axis Flyout
    • add separate github workflow for python linting

v0.14.0

  • educational materials
    • add Cognitive Service fairness assessment notebooks
  • new features

    • Error Analysis

      • add new accuracy and f1-score metrics in erroranalysis python backend and UI frontend
      • add importances to ErrorReport for error analysis
      • make ErrorReport, PredictionsAnalyzer and ModelAnalyzer public in erroranalysis package
      • display message bar after saving a cohort in error analysis dashboard

    • Counterfactual

      • implement save and load for counterfactual manager
      • make load method uniform across all RAI managers
      • perform schema validations on serialized counterfactual output

    • Interpret

      • upgrade interpret-community to 0.22.0
  • bug fixes

    • Model Analysis

      • remove drag and drop instructions in the dashboard navigation
      • add theme support for model statistics and dataset explorer
      • remove setup_explainer after ModelAnalysis load call in tests
      • add theme support for feature importances, causal, counterfactual tabs
      • hide opened feature list when clicking on the heatmap tab in model analysis
      • add localization for delete cohort dialog

    • Interpret

      • set is_run class variable in explainer manager

    • Causal

      • add init file to causal schema directory

    • Error Analysis

      • fix color for metrics that are not error metrics to be green in legend and heatmap nan cells
      • update features list in static view to have apply button removed and labels disabled for tree view parameters
      • update features list text for static case
      • fix running what if on categorical numeric values in error analysis dashboard
      • add theme support for error analysis view
      • fix for importances in static view causing failures when unspecified
      • fix feature list alignment issue

  • other

    • add documentation for the release process
    • add E2E test for model analysis, running from python jupyter notebook and validating in cypress
    • sort all python imports using isort
    • add aggregate feature importance tests for responsibleaitoolbox
    • add E2E individual feature importance tests for responsibleaitoolbox-classification-model-assessment notebook
    • fix image links in responsibleaitoolbox-classification-model-assessment.ipynb notebook to reference images in folder instead of using embedded images directly
    • rename causal tests

v0.13.0

  • new features

    • Model Analysis

      • add "Save amd switch" button to cohort creation
      • pass classes parameter to both error analysis dashboard and error analysis manager in model analysis
      • upgrade interpret-community to 0.20.0

    • Error Analysis

      • update erroranalysis to 0.1.24
  • bug fixes

    • Model Analysis

      • fix issue with widgets rendering with 8px height

    • Error Analysis

      • fix precision and recall score failing on string labels in error analysis
      • fix multiclass precision and recall calculation for NA zero values in heatmap
      • edit string in error analysis cohort creation
      • remove text in heatmap suggesting to select features when in static view
      • minor doc fix for examples in error analysis dashboard

    • Counterfactual

      • fix horizontal scroll bar disappears for small screen in what-if panel

    • Fairness

      • fix data issue caused robot page exception on fairnessDetails
  • other
    • fix broken link to getting started notebook
    • fix broken image link on main readme
    • fix the issues with sample notebooks markdown images
    • delete old notebooks
    • change tests to run on new notebooks, add new notebook to tests

v0.12.1

  • new features

    • Error Analysis

      • add disabled tree view and heatmap for static view
  • bug fixes

    • Error Analysis

      • fix class length causing labels to not be shown in simple view of Error Analysis
      • fix cohort filter tooltip behind cohort info panel

    • Causal

      • fix trace0 appearing on hover tooltip in plots

    • Counterfactual

      • handle cases where counterfactuals failed to compute
      • add tooltip for counterfactual info
  • other
    • some grammatical errors were corrected on documentation page

v0.12.0

  • new features

    • Model Analysis

      • add global cohort statistics

    • Error Analysis

      • add metric selector dropdown
      • add tree view parameters as sliders to features selector panel
      • add tooltip to heatmap categories and left-justify the heatmap

    • Counterfactual

      • upgrade dice-ml to 0.7.2
  • bug fixes

    • Model Analysis

      • fix cohort information in heatmap resizing in model analysis
      • log warning when regression model has predict_proba()

    • Error Analysis

      • fix error rate gradient for classification scenario
      • fix quantile binning toggle should stay on when making actions
      • fix multiple issues in instance view, including:
        • when selecting multiple rows in correct and incorrect panels, and going to all selected, if a point is removed in the middle all of the points below get removed
        • when removing points in the all selected panel, the count does not get updates in the correct/incorrect panels until they are selected
        • when using what if, make points not selectable
      • fix precision and recall calculation in the UI changing with number of bins
      • decrease the lower bound on first category for quantile binning
      • fix handling of numeric features names in pandas for error analysis tree view
      • reverse y axis in heatmap so that higher values are towards the top of the matrix
      • add warnings for 2d heatmap case and test for quantile binning with duplicate edges

    • Causal

      • sort causal data point by effect size
      • set treatment list spin range
      • fix causal global effect plot x-axis titles cut off
      • fix causal categorical treatment average gains text

    • Counterfactual

      • change current class label for regression task
      • raise exception when users configures less than required counterfactuals
      • show options for categorical features on counterfactual list

    • Fairness

      • add error handling for missing metrics in fairness
  • other
    • remove Axios library from dependencies
    • rename wrapped-flask to rai_core_flask
    • remove deprecated babelconfig option in workspace.json

v0.11.0

  • new features
    • table for list of available cohorts
    • added more metrics to ErrorAnalysisDashboard, including precision, recall and mean abs error
  • bug fixes
    • fix various UI usability bugs
    • fix for clicking custom scatter plot point

    • Error Analysis

      • fix selecting what if datapoint crashing dashboard
      • fix error in quantile binning when there are too many duplicate points
      • fix displayed error rate in all data cohort in cohort list
      • fix error in heatmap for classification case
      • fix features list not getting disabled in static view
      • update description text for static view
  • other
    • improve validation for model prediction
    • reorder counterfactual table columns
    • add absolute sorting for local importance charts
    • improve spacing and fonts across dashboard
    • fix typos in docstrings
    • improve wording of text on dashboard
    • add warnings for soft data limits
    • remove font override for all components
    • add code coverage to repository

v0.10.0

  • educational materials

    • update current model analysis classification notebook by adding treatment features for causal analysis

  • new features

  • breaking changes

  • bug fixes

    • Model Analysis:

      • change icon for ViewList in main menu
      • fix model statistics component not showing all cohorts
      • put causal component last, remove navigation
      • hide removal dialog on click
      • move cohort info and button to sticky menu, replace error stats with filters

    • Error Analysis:

      • refactor heatmap code and fix display issues for categoricals
      • fix incorrect values in heatmap when there are empty error cells
      • added quantile binning and number of bins functionality to erroranalysis package
      • added quantile binning and number of bins to UX and hooked up calls to backend
      • added tooltip to list of features for tree view
      • fix backend to support equals filter for integer categorical columns
      • fix backend to support include and exclude filters for integer categorical columns
      • update default value for matrix feature
      • fix cohort when switching tree view

    • Causal:

      • remove unnecessary z-index on what-if and causal components
      • update causal policy to support categorical features
      • format policy tree categorical features
      • fix description on how confounding features are handled
      • use lighter lines for causal plotly point graphs

    • Fairness:

      • incorporate uncertainty quantification in fairness widget v2

    • Counterfactuals:

      • upgrade dice-ml to 0.7
      • update counterfactual list to allow string values
      • extend unseen categorical check to counterfactuals
    • add dropdown to dataset explorer by default
    • fix removing what if point will keep a faded start on the chart
    • suppress distracting logging from flask when running widgets
    • fix large x and y axis going out of bounds
    • refactor multiple instances of convert to list method in several dashboards
    • add missing interpretability information and guidelines

  • other

    • fix message and check for environment in release pipeline

v0.9.4

  • bug fixes
    • fixed causal treatment policy display issue happening when switching between features

v0.9.3

  • educational materials
  • new features
    • causal treatment cost argument can be passed per feature/sample/category
    • target added to counterfactual tabular output
    • causal analysis now trains the model on training data, not both train and test
    • min_child_samples parameter for error analysis
  • breaking changes
  • bug fixes
    • improved text for causal what-if
    • fix for missing values in dataset
    • improvement for default explanation tab
    • fix for multiple feature importances detail view
    • fix for causal what-if slider being stuck
    • fixes for cohort selection
    • fixes for error analysis matrix view
    • fix for extra scroll bar in counterfactuals view
    • fix for train data input in classification notebook
    • fix for bug in counterfactual feature_to_vary parameter
    • fix for x and y axes of error analysis heatmap
    • fix for counterfactual predict
  • other
    • improved error messages for responsibleai ModelAnalysis
    • improved validation when not passing a model to ModelAnalysis
    • updated ModelAnalysis classification notebook to use separate train and test datasets

v0.9.2

  • educational materials
  • new features
    • Model Analysis:
      • add search box to counterfactuals panel
      • add support for feature to vary and permitted range in counterfactual manager
    • add matrix features and tree features to error report object in erroranalysis package
  • breaking changes
  • bug fixes
    • call flask service credential
    • Model Analysis:
      • fix hardcoded feature name in counterfactual panel
      • fix refresh on global cohort change
      • add tree_features and matrix_features params to static tree and heatmap
      • update sticky menu
      • change counter factual desired_range to tuple
    • fix aggregate feature importance chart for multiclass classification
    • fix serialize_json_safe not escaping special characters which caused dashboard to fail to render on pandas dataframe that had double quotes in string values
  • other
    • add dice-ml to intersphinx_mapping
    • add missing documentation for counterfactual manager
    • fix python responsibleai package test for windows

v0.9.1

  • educational materials
  • new features
  • breaking changes
    • error report in erroranalysis package had json_tree renamed to tree and json_matrix renamed to matrix
  • bug fixes
    • fix econML inputs
    • add categorical feature to causal whatif
    • fix counterfactual panel scroll bar
    • round precision in causal analysis tables
    • fix calculating feature importances after error report due to dataframe on analyzer being modified
  • other
    • dump information about the most time consuming tests

v0.9.0

  • educational materials
    • add Model Analysis notebook for regression
  • new features
    • local importance chart for counterfactual dashboard
    • add multi causal policy support
    • add delete tab dialog and remove inline widget for model assessment dashboard
    • add treatment policy and table to causal analysis dashboard
    • create new custom individual feature importance component
  • breaking changes
  • bug fixes
    • Model Analysis:
      • sticky layer for cohort and dashboard settings
      • restrict EA tree to 80% width
      • restrict ipykernel<0.6
      • fix flyout title and description
      • fix add button behavior by adding components in the appropriate spot
      • show add button only for components that can render
      • add validations for input parameters for Model Analysis class
      • validate treatment features in causal manager
      • explicitly pass model task to MimicExplainer in model analyzer
    • Error Analysis:
      • remove 100k row limit
      • fix zero nodes in tree view
      • fix search functionality in features list
      • fix displayed filter order in tree view to start from root node
      • raise warning for older version of pandas when feature names contain '-'
      • expand number of decimals shown in filter tooltips when values below 0.1
      • fix regression case for 1d heatmap matrix metrics being calculated incorrectly
    • set original data from dropdown for counterfactuals
    • use test data instead of train data for error analysis manager in model analyzer
    • causal analysis to keep all levels in dataframe outputs
    • pin networkx pip package to prevent installation of matplotlib
    • add target_column causal policy gain to UI control
  • other
    • unit tests for multiclass classification for causal analysis
    • add model analysis notebooks to notebook gate
    • add test for causal manager for None categoricals
    • fix example code for model assessment

v0.8.0

  • educational materials
  • new features
    • enabled categoricals on MimicExplainer for the explainer manager
    • updated causal constants to enable linear models and allow high cardinality features by default
    • support newest policy output format from econml
  • breaking changes
    • continuous_features in CounterfactualManager.add() is deprecated in favor of categorical_features in ModelAnalysis
  • bug fixes
    • fixed error analysis add after model analysis deserialization
  • other
    • improved tests for visualization dependencies

v0.7.0

  • educational materials
  • new features
    • added widget serialization for causal analysis
  • breaking changes
  • bug fixes
  • other

v0.6.1

  • educational materials
  • new features
  • breaking changes
  • bug fixes
  • other

v0.6.0

  • educational materials
  • new features
  • breaking changes
  • bug fixes
  • other

v0.5.0

  • educational materials
    • cleanup commented out explainers from interpretability notebook
  • new features
    • added support for causal, counterfactual, and error analysis tools in responsibleai
    • added model analysis widget to raiwidgets
    • added support for regression to error analysis
    • added policy tree to causal manager
  • breaking changes
    • renamed package raitools to responsibleai
    • renamed top-level class RAIAnalyzer to ModelAnalysis
    • removed support for fairness in responsibleai package
  • bug fixes
    • fixed bug in precision statistic calculation in dashboard
    • fixed bug in loading explanations with ModelAnalysis
    • fixed on predict method to use correct dtypes
    • fixed individual importances chart when no data available
  • other
    • create combined feature importances tab for local and global explanation in model assessment
    • add section headers to model assessment dashboard
    • add manual cohort creation to the model assessment dashboard

v0.4.0

  • educational materials
    • cleanup commented out explainers from interpretability notebook
  • new features
    • fix databricks environment with raiwidgets dashboards
    • updates to ErrorAnalysisDashboard:
      • improve the features list to use DetailsList
      • added tooltips to legend
      • initial static view for error analysis, specifically matrix filter and tree view
      • fixed tooltip node hover
      • fix indexing issue on cohort shift
      • add new simple dashboard implementation, which only takes in predictions and does not need model or dataset
    • updates to ModelAssessmentDashboard:
      • set up tabs on the left side
      • add model statistics tab
      • add pivot for error analysis
      • remove fullscreen button
    • refactor error analysis python code into separate common package
  • breaking changes
  • bug fixes
    • reduce the lower-bound on the lightgbm dependency
    • update greenlet and gevent dependencies to fix releases
  • other
    • use global context in interpret, error-analysis, and model-assessment
    • moved dataset explorer into a new project dataset-explorer
    • created new projects causality and counterfactuals
    • remove fairness v1 dashboard

v0.3.1

  • Bug fix:
    • fix public VM failing requests in rai-core-flask package, update raiwidgets to latest rai-core-flask package

v0.3.0

  • Bug fixes:
    • Fix all data cohort always appearing first in cohorts lists
    • Add number selected text under local explanation radio buttons
    • Fix the issue that categorical feature is not able to change in cohort editor.
    • Fix categorical feature in what-if tab

v0.2.2

  • Bug fix:
    • Fix categorical feature in ice plot
    • Left align tabs in explanation window of error analysis
    • Move the selected features and feature importances inside a scrollable pane
    • Show aggregate plots by default in dataset explorer
    • Fix indexing issue in heatmap when selecting all cells

v0.2.1

  • educational materials:
    • add imports for MimicExplainer and PFIExplainer in the interpretability-dashboard-employee-attrition notebook
  • features
    • migrate cohort and dataset utilities to core-ui
    • remove circular dependencies
    • add confirmation popup in ExplanationDashboard when cancelling editing a cohort.
    • fix clipped dropdown in FairnessDashboard
    • keep FairnessDashboard chart state in single model view
    • updates to ErrorAnalysisDashboard:
      • in large data view, change cohort state to use full data instead of downsampled explanation data
      • add true y and predicted y values
      • fix categorical labels handling in what if panel
      • add breadcrumb to error explorer view and minor style adjustments to navigation panel
      • fix indexing issues for all data cohort
      • add color column to local explanation view

v0.2.0

This is the first release of the raiwidgets package.

  • educational materials:
    • notebook analyze-categoricals-binary-classification-local.ipynb added
    • notebook erroranalysis-pretability-dashboard-breast-cancer.ipynb added
    • notebook erroranalysis-interpretability-dashboard-census.ipynb added
    • notebook fairness-credit.ipynb added
    • notebook fairness-dashboard-loan-allocation.ipynb added
    • notebook fairness-interpretability-dashboard-loan-allocation.ipynb added
    • notebook interpretability-dashboard-employee-attrition.ipynb added
  • features
    • FairnessDashboard changes compared to fairlearn's FairlearnDashboard which was migrated to raiwidgets:
      • Restructured performance metric selection as list rather than set of tiles for easier navigation.
      • Added fairness metric selection using a grouped list.
      • Added dropdowns to model comparison view and single model view to allow users to quickly switch between sensitive features, performance metrics, and fairness metrics.
      • Added metric table to single model view.
      • Added dropdown to switch between charts in single model view.
      • Simplified API to extract sensitive feature names directly from sensitive_features.
    • ErrorAnalysisDashboard: first release
    • migrated ExplanationDashboard from interpret-community to raiwidgets

We're skipping v0.1.0 to get all our packages onto the same version:

  • pypi:

    • raiwidgets: first real release

  • npm:

    • @responsible-ai/fairness: previously 0.1.*
    • @responsible-ai/interpret: previously 0.0.*
    • @responsible-ai/error-analysis: first real release
    • @responsible-ai/mlchartlib: previously 0.0.*
    • @responsible-ai/core-ui: previously 0.0.*
    • @responsible-ai/localization: previously 0.0.*