Fix sort by absolute value from local importance chart #1299
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## main #1299 +/- ##
=======================================
Coverage 86.13% 86.13%
=======================================
Files 99 99
Lines 4509 4509
=======================================
Hits 3884 3884
Misses 625 625
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report at Codecov.
|
imatiach-msft
approved these changes
Mar 29, 2022
19 tasks
gaugup
added a commit
that referenced
this pull request
Apr 21, 2022
…aking.ipynb (#1252) * [WIP] Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix the e2e test for notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix cohort namespace Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * minor fix to url for responsibleai package in setup.py (#1260) * Counterfactual Chart: Correct target description according to task_type (#1261) * Counterfactual Chart: Correct target description according to task_type Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change function name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make Range lowercase Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix whitespace in values of adult census income dataset (#1263) * Add what-If scatter chart from highchart lib (#1262) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * add a builddebug yarn command to build UX locally which can be debugged in browser e2e (#1265) * allow rai text insights to work with RAI dashboard (#1269) * remove duplicate code in explanation dashboard (#1266) * Individual causal style responsive (#1268) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * Causal Style * Allow duplicating cohorts multiple times (#1274) * allow duplicating a cohort more than once * lintfix * Disable column header highlighting on hover in IndividualFeatureImportanceView (#1272) * disable column highlight on hover * lintfix * Rename new cohorts from "Unsaved" to "Temporary cohort" (#1273) * rename Unsaved to Temporary cohort * localize temp cohort * Counterfactual style refactor (#1275) * style refactor * test * test * test * fix * Don't change cursor on hover over cohort name * Fix (#1281) * fix cohort info styling (#1277) * fix readme link to fairness and interpretability example notebook (#1282) * add new RAI Utils package for common utilities shared across RAI packages (#1280) * Add ICE chart (#1283) * Fix * ice chart * ic * test * test * update docstring for explanation dashboard in regards to min number of rows (#1271) * make builds more reliable by adding retry logic to urlretrieve calls in notebooks (#1218) * upgrade pytest to 7.0.1, remove mock and updgrade pytest-mock to 3.6.1 (#1287) * remove deprecated codecov parameter (#1293) * Fix min/max special case in cohort filter creation with "in the range of" (#1279) * fix logic in the case that min or max are zero * lintfix * Rename 'Dashboard navigation' to 'Dashboard configuration' (#1291) * Rename 'Dashboard navigation' to 'Dashboard configuration' Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add raiutils to PR template (#1290) * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix linting Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix flake8 errors Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * attempt to fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove check Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](moment/moment@2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <xuke@microsoft.com> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <xuke@microsoft.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <xuke@microsoft.com> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <xuke@microsoft.com> * remove dup Signed-off-by: Ke Xu <xuke@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * update version to match studio (#1357) Signed-off-by: Ke Xu <xuke@microsoft.com> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more tests cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohorts filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic data validation checks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove commented out code Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add handling of regression filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohortData test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * isort fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * REmove notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix typescript build Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code issues Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix eslint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: tongy-msft <91754176+tongyu-microsoft@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com>
gaugup
added a commit
that referenced
this pull request
May 26, 2022
…aking.ipynb (#1252) * [WIP] Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix the e2e test for notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix cohort namespace Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * minor fix to url for responsibleai package in setup.py (#1260) * Counterfactual Chart: Correct target description according to task_type (#1261) * Counterfactual Chart: Correct target description according to task_type Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change function name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make Range lowercase Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix whitespace in values of adult census income dataset (#1263) * Add what-If scatter chart from highchart lib (#1262) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * add a builddebug yarn command to build UX locally which can be debugged in browser e2e (#1265) * allow rai text insights to work with RAI dashboard (#1269) * remove duplicate code in explanation dashboard (#1266) * Individual causal style responsive (#1268) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * Causal Style * Allow duplicating cohorts multiple times (#1274) * allow duplicating a cohort more than once * lintfix * Disable column header highlighting on hover in IndividualFeatureImportanceView (#1272) * disable column highlight on hover * lintfix * Rename new cohorts from "Unsaved" to "Temporary cohort" (#1273) * rename Unsaved to Temporary cohort * localize temp cohort * Counterfactual style refactor (#1275) * style refactor * test * test * test * fix * Don't change cursor on hover over cohort name * Fix (#1281) * fix cohort info styling (#1277) * fix readme link to fairness and interpretability example notebook (#1282) * add new RAI Utils package for common utilities shared across RAI packages (#1280) * Add ICE chart (#1283) * Fix * ice chart * ic * test * test * update docstring for explanation dashboard in regards to min number of rows (#1271) * make builds more reliable by adding retry logic to urlretrieve calls in notebooks (#1218) * upgrade pytest to 7.0.1, remove mock and updgrade pytest-mock to 3.6.1 (#1287) * remove deprecated codecov parameter (#1293) * Fix min/max special case in cohort filter creation with "in the range of" (#1279) * fix logic in the case that min or max are zero * lintfix * Rename 'Dashboard navigation' to 'Dashboard configuration' (#1291) * Rename 'Dashboard navigation' to 'Dashboard configuration' Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add raiutils to PR template (#1290) * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix linting Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix flake8 errors Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * attempt to fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove check Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](moment/moment@2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <xuke@microsoft.com> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <xuke@microsoft.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <xuke@microsoft.com> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <xuke@microsoft.com> * remove dup Signed-off-by: Ke Xu <xuke@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * update version to match studio (#1357) Signed-off-by: Ke Xu <xuke@microsoft.com> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more tests cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohorts filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic data validation checks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove commented out code Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add handling of regression filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohortData test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * isort fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * REmove notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix typescript build Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code issues Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix eslint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: tongy-msft <91754176+tongyu-microsoft@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com>
gaugup
added a commit
that referenced
this pull request
Jun 6, 2022
* Add more utilities into raiutils Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix faulty file name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix linting Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix flake8 errors Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * attempt to fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove check Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](https://github.com/moment/moment/compare/2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <xuke@microsoft.com> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <xuke@microsoft.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <xuke@microsoft.com> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <xuke@microsoft.com> * remove dup Signed-off-by: Ke Xu <xuke@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * update version to match studio (#1357) Signed-off-by: Ke Xu <xuke@microsoft.com> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more tests cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohorts filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic data validation checks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove commented out code Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add handling of regression filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohortData test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * isort fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * REmove notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix typescript build Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code issues Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix eslint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> * Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb (#1252) * [WIP] Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix the e2e test for notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix cohort namespace Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * minor fix to url for responsibleai package in setup.py (#1260) * Counterfactual Chart: Correct target description according to task_type (#1261) * Counterfactual Chart: Correct target description according to task_type Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change function name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make Range lowercase Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix whitespace in values of adult census income dataset (#1263) * Add what-If scatter chart from highchart lib (#1262) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * add a builddebug yarn command to build UX locally which can be debugged in browser e2e (#1265) * allow rai text insights to work with RAI dashboard (#1269) * remove duplicate code in explanation dashboard (#1266) * Individual causal style responsive (#1268) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * Causal Style * Allow duplicating cohorts multiple times (#1274) * allow duplicating a cohort more than once * lintfix * Disable column header highlighting on hover in IndividualFeatureImportanceView (#1272) * disable column highlight on hover * lintfix * Rename new cohorts from "Unsaved" to "Temporary cohort" (#1273) * rename Unsaved to Temporary cohort * localize temp cohort * Counterfactual style refactor (#1275) * style refactor * test * test * test * fix * Don't change cursor on hover over cohort name * Fix (#1281) * fix cohort info styling (#1277) * fix readme link to fairness and interpretability example notebook (#1282) * add new RAI Utils package for common utilities shared across RAI packages (#1280) * Add ICE chart (#1283) * Fix * ice chart * ic * test * test * update docstring for explanation dashboard in regards to min number of rows (#1271) * make builds more reliable by adding retry logic to urlretrieve calls in notebooks (#1218) * upgrade pytest to 7.0.1, remove mock and updgrade pytest-mock to 3.6.1 (#1287) * remove deprecated codecov parameter (#1293) * Fix min/max special case in cohort filter creation with "in the range of" (#1279) * fix logic in the case that min or max are zero * lintfix * Rename 'Dashboard navigation' to 'Dashboard configuration' (#1291) * Rename 'Dashboard navigation' to 'Dashboard configuration' Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add raiutils to PR template (#1290) * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix linting Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix flake8 errors Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * attempt to fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove check Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](https://github.com/moment/moment/compare/2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <xuke@microsoft.com> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <xuke@microsoft.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <xuke@microsoft.com> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <xuke@microsoft.com> * remove dup Signed-off-by: Ke Xu <xuke@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * update version to match studio (#1357) Signed-off-by: Ke Xu <xuke@microsoft.com> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more tests cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohorts filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic data validation checks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove commented out code Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add handling of regression filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohortData test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * isort fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * REmove notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix typescript build Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code issues Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix eslint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: tongy-msft <91754176+tongyu-microsoft@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> * Add more context to download print in fetch_dataset (#1368) * Add missing desired range to counterfactual chart (#1366) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a summary for disaggregated analysis showing ratio and difference (#1367) * add fairness metric table, update existing tables (order, N/A values, grid axis label lengths) * lintfix * merge fixes * addressed eslint issue for null value by creating a dedicated file, localized N/A, updated tooltip text to be wrapped (+ associated unit tests) * keep feature flag * Update PR template to be more intuitive (#1116) * Updating PR template to make sure there's at least 1 box that can be ticked. * replace checkboxes with GitHub emojis for checkmark and x to prevent GitHub from interpreting it as a task list * update PR template based on feedback * lintfix * Update CODEOWNERS file (#1369) * Use global cohort for disaggregated analysis (#1370) * use global cohort for disaggregated analysis and warn user about it * lintfix * Change procedural loops to functions map() and filter() (#1255) * Change procedural loops to function map() and filter() Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix review comment Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add accuracy as multiclass metric (#1371) * support accuracy for multiclass classification * lintfix * update code owner (#1372) Signed-off-by: Ke Xu <xuke@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Rename _validate_model_analysis_input_parameters() to _validate_rai_insights_input_parameters (#1373) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Cache and serialize the counterfactual explainer (#1374) * Cache and serialize the counterfactual explainer Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a top-level pivot to model overview (#1375) * show only disaggregated analysis OR only dataset cohorts * lintfix * basec -> basic typo * Add a toggle to model overview to turn heatmap colors on and off (#1376) * add visual display toggle * lintfix * simplify onVisualDisplayToggleChange method * color replacements * fluentui colors * localization, style file * lintfix * Create rai-e2e package for tests shared with AML (#1378) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * index update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * remove files from wid-e2e Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update datashape Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * add pyspark support to tree surrogate model in error analysis (#1251) * Create e2e package for tests shared with AML (#1383) * new buildable folder Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * remove rai-e2e lib Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * add license Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Add feature and metric configuration flyouts to model overview (#1381) * show only disaggregated analysis OR only dataset cohorts * lintfix * basec -> basic typo * feature config * merge fixes * metric config * lintfix * wrap metric descriptions * lintfix * move constant to constants file to avoid circular dependency, lintfix * move inline code into functions * Add probability distribution line chart to model overview (#1382) * line chart for prob distribution * lintfix * fix race condition in tree view update code which can cause root stats to be ignored on selected cohort in RAI dashboard static view (#1386) * String updates for model overview (#1387) * string updates for model overview * lintfix * Handle the loading of causal models more gracefully (#1377) * Handle the loading of causal models more gracefully Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review commensts and fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comment Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Some fixes for data explorer, heatmap, counterfactual components (#1389) * add x axis and update heatmap * cf custom point color should be consistent * cf custom point name should be displayed * disable causal treatment whatif in static view (#1391) * fix big data set for counter factual chart (#1393) Signed-off-by: Ke Xu <xuke@microsoft.com> * alignment fixes for causal (#1392) * Fix feature selection state carry-over bug in model overview (#1390) * Provide feature flight support to dashboard, test environment, and Python widget (#1385) * Python part of exposing flights * add feature flighting * fix path for test case * update snapshot * add none constant, contrib guide update, and unit tests for feature_flights param * flake8 * isort * release raiwidgets and responsibleai v0.18.1 (#1413) * Add warning in counterfactual manager when unable to load explainer (#1412) * Add warning in counterfactual manager when unable to load explainer Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix linting Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bug fixes on 'Set value' not copying over feature values correctly in what if counterfactual panel (#1416) * fix cf set value bug * remove comment * fix error on machines with pyspark installed where passed dataframe is not spark pandas (#1415) * add postbuild branch trigger (#1417) * Fix causal UI strings according to classification/regression tasks (#1419) * Fix causal UI strings according to classification/regression tasks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint error Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix UI test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix description for model overview (#1425) * fix description for model overview * keep new description for new model overview * fix failing to create error report when filter_features is empty list (#1421) * filter out missing values from what if dropdown to prevent explanation dashboard from crashing (#1418) * Remove |Set Value| blurb in case it is not availble in counterfactual panel (#1426) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add y-axis description to counterfactual feature importance chart (#1423) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> * Add the user class name to causal UI strings (#1422) * Fix causal UI strings according to classification/regression tasks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint error Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix UI test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add the user class name to causal UI strings Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> * fix math.min / max for array size more than 10^7 (#1427) Signed-off-by: Ke Xu <xuke@microsoft.com> * upgrade pytest and lightgbm to try to fix random pytest segfault test failures (#1424) s * fix flaky notebook causing build failures by adding retry logic (#1431) * Upper bound SciKit-Learn to address freeze in causal (#1432) <!--- Provide a general summary of your changes in the Title above --> ## Description Replaces #1429 to address #1430 . Causal analysis is getting stuck with the latest release of SciKit-Learn. This contains: - Test case which gets stuck with SciKit-Learn 1.1.0 - Upper bound on SciKit-Learn in `requirements.txt` ## Checklist <!--- Make sure to satisfy all the criteria listed below. --> - [x] I have added screenshots above for all UI changes. - [x] Documentation was updated if it was needed. - [x] New tests were added or changes were manually verified. Signed-off-by: Richard Edgar <riedgar@microsoft.com> * fix dependency chart axis updating with incorrect values in explanation dashboard (#1437) * fix codecov and widget test screenshot uploads (#1428) * release raiwidgets and responsibleai v0.18.2 (#1439) * fix (#1441) Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Fix cohort name conflict and not run few tests for AML (#1442) * fix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Few e2e tests changes to accommodate AML static tests (#1445) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Fix locators logic for string features - data explorer and model statistics components (#1446) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * fix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * fix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Add more unittests RAI dashboard input class (#1448) * Add unit tests for ResponsibleAIDashboardInput Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: tongy-msft <91754176+tongyu-microsoft@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> Co-authored-by: Richard Edgar <riedgar@microsoft.com>
ms-kashyap
pushed a commit
that referenced
this pull request
Jun 7, 2022
* Add more utilities into raiutils Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix faulty file name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix linting Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix flake8 errors Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * attempt to fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove check Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](https://github.com/moment/moment/compare/2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <xuke@microsoft.com> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <xuke@microsoft.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <xuke@microsoft.com> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <xuke@microsoft.com> * remove dup Signed-off-by: Ke Xu <xuke@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * update version to match studio (#1357) Signed-off-by: Ke Xu <xuke@microsoft.com> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more tests cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohorts filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic data validation checks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove commented out code Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add handling of regression filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohortData test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * isort fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * REmove notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix typescript build Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code issues Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix eslint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> * Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb (#1252) * [WIP] Add pre-defined cohorts in responsibleaidashboard-diabetes-decision-making.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix the e2e test for notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix cohort namespace Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * minor fix to url for responsibleai package in setup.py (#1260) * Counterfactual Chart: Correct target description according to task_type (#1261) * Counterfactual Chart: Correct target description according to task_type Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change function name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Make Range lowercase Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix whitespace in values of adult census income dataset (#1263) * Add what-If scatter chart from highchart lib (#1262) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * add a builddebug yarn command to build UX locally which can be debugged in browser e2e (#1265) * allow rai text insights to work with RAI dashboard (#1269) * remove duplicate code in explanation dashboard (#1266) * Individual causal style responsive (#1268) * add whatIf scatter chart * widget test * what if local importance bar chart * fix * widget * fix tooltip * refactor * test * test * Causal Style * Allow duplicating cohorts multiple times (#1274) * allow duplicating a cohort more than once * lintfix * Disable column header highlighting on hover in IndividualFeatureImportanceView (#1272) * disable column highlight on hover * lintfix * Rename new cohorts from "Unsaved" to "Temporary cohort" (#1273) * rename Unsaved to Temporary cohort * localize temp cohort * Counterfactual style refactor (#1275) * style refactor * test * test * test * fix * Don't change cursor on hover over cohort name * Fix (#1281) * fix cohort info styling (#1277) * fix readme link to fairness and interpretability example notebook (#1282) * add new RAI Utils package for common utilities shared across RAI packages (#1280) * Add ICE chart (#1283) * Fix * ice chart * ic * test * test * update docstring for explanation dashboard in regards to min number of rows (#1271) * make builds more reliable by adding retry logic to urlretrieve calls in notebooks (#1218) * upgrade pytest to 7.0.1, remove mock and updgrade pytest-mock to 3.6.1 (#1287) * remove deprecated codecov parameter (#1293) * Fix min/max special case in cohort filter creation with "in the range of" (#1279) * fix logic in the case that min or max are zero * lintfix * Rename 'Dashboard navigation' to 'Dashboard configuration' (#1291) * Rename 'Dashboard navigation' to 'Dashboard configuration' Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add raiutils to PR template (#1290) * fix heatmap bug (#1297) * Make "save and switch" work from cohort settings (#1276) * make save and switch work * fix naming * lintfix * adjustment according to Ilya's comment * lintfix * add retry logic to codecov step and only upload results for one python version (#1298) * add github action to release raiutils to pypi (#1294) * Add highchart for Dataset Explorer (#1286) * test * style * click * fix test * fix test * test * test * test * test * Update requirements-linting.txt to add flake8-pytest-style (#1296) * Fix sort abs (#1299) * Rename "base cohort" to "global cohort" (#1278) * change base cohort to global cohort * fix spelling * lintfix * fix codecov comment not appearing on PRs (#1302) * take absolute value of error calculation for regression scenario (#1301) * Limit individual feature importance selection to up to 5 (#1305) * update feature importance string * limit selection to up to 5 * add group count * remove message bar, show info icon instead * update e2e locator * fix E2E failure on feature importance * add ariaLabel for expand collapse button * add renderOnNewLayer props * Add error message for counterfactual panel (#1310) * add error message for counterfactual * update error message in camel case to fix build error * Add to_json() and from_json() methods to Cohort class (#1300) * Add to_json() and from_json() methods to Cohort class Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix linting Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a highchart heatmap helper class (#1307) * add highchart heatmap helper class * add erroneously deleted line back * Fix cohort setting string (#1304) * Fix string * remove none * name * test * Fix all data style (#1303) * Add a feature flag for the new model overview experience (#1306) * add feature flag for new model overview experience and turn it off by default * remove useless constructor * Clean up charts code (#1313) * clean up chart code * remove arg * Expand the counterfactual flyout to cover the full page (#1315) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump minimist from 1.2.5 to 1.2.6 (#1292) * Bump minimist from 1.2.5 to 1.2.6 Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * minimist ^1.2.6 Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * fix random node download failures by upgrading to latest github action with retry logic (#1317) * Add dataset cohort table to new ModelOverview experience (#1314) * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * Add installation instructions for raiwidgets to README (#1320) * refactor RAIInsights into RAIInsightsBase class for basic functionality (#1284) * Fix what if counterfactual header and description text misaligned (#1316) * align * e2e * add clear temporary cohort button to error analysis (#1322) * Raise UserConfigValidationException in case no model but valid model serializer (#1325) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add test case for handling different types in causal (#1321) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * show shift to an empty cohort in tree view as an empty node (#1318) * Bug fixing (#1326) * Move chart description up and remove scroll bar * Change string * Add box outlier for dataset explorer (#1323) * add outlier for dataset explorer * name * update string when no datapoint selected (#1331) * Fix Big empty space for featureImportance chart (#1328) * legend * removed invalid test case * constant * Disable save as new cohort button if nothing is selected in error tree (#1327) * Add disaggregated analysis table/heatmap (#1332) * disaggregated analysis changes only * lintfix * Change warning message to user exception for model type and task type mismatch (#1330) * Change warning message to user exception for model type and task type mismatch Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix flake8 errors Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change the counterfactual text color from black to grey (#1337) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Limit each component description width up to 750px for readability (#1336) * limit description width up to 750px * export maxWidth from a common place * block empty cohort creation in RAI Dashboard (#1335) * Add warning message in cohort editor for invalid input value; Update 'Shift cohort' to 'Switch cohort' (#1339) * add error message for invalid value * update shift cohort to switch cohort * Rename counterfactual style files to confirm with *.styles.ts (#1338) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add disaggregated analysis table to Model Overview (#1341) * pull in changes for disaggregated analysis * add styles file * add textured NaN cells * module import for textured cells and grid y axis * lintfix * use combobox for dropdown rather than dropdown * lintfix * Rename causal style files to confirm with *.styles.ts (#1342) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * update responsibleai to interpret-community 0.25.0 (#1343) * All component title and descriptions should be aligned (#1346) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Remove 5K limit blurb from local explanations tab (#1347) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Sort features by default in counterfactual flyout (#1312) * Sort features by default in counterfactual flyout Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * attempt to fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove check Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bump moment from 2.28.0 to 2.29.2 (#1333) Bumps [moment](https://github.com/moment/moment) from 2.28.0 to 2.29.2. - [Release notes](https://github.com/moment/moment/releases) - [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md) - [Commits](https://github.com/moment/moment/compare/2.28.0...2.29.2) --- updated-dependencies: - dependency-name: moment dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Counterfactual flyout top section need to be moved to left & Error analysis move side content to align with description text (#1350) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1344) * update error analysis documentation to clarify the error tree splits on errors even when other metrics are selected (#1349) Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> * update erroranalysis to 0.2.1 and remove some duplicate dependencies (#1334) * disable turbo checking for large amount of data (#1351) Signed-off-by: Ke Xu <xuke@microsoft.com> * force re-render when chart type changes (#1354) Signed-off-by: Ke Xu <xuke@microsoft.com> * move the root all data statistics to ErrorReport and ErrorAnalysisData (#1352) * Rename output column name in the counterfactual flyout (#1353) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Show column chart for categorical feature in data explorer (#1355) * Show column chart for categorical feature in data explorer * address comments * update fluentui (#1356) Signed-off-by: Ke Xu <xuke@microsoft.com> * update code owner (#1308) * update code owner Signed-off-by: Ke Xu <xuke@microsoft.com> * remove dup Signed-off-by: Ke Xu <xuke@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * update version to match studio (#1357) Signed-off-by: Ke Xu <xuke@microsoft.com> * alignment fixes (#1359) * Add charts for new model overview experience (#1348) * rename OverallTable to DisaggregatedMetricsTable and move to core-ui * Copy the ModelPerformanceTab into model-assessment and rename to ModelOverview * reference OverallTable again in fairness widget * refactor core chart component out into core-ui * refactor out core chart component into core-ui * lintfix * undo tsconfig.lib.json change * fix chartAndConfigsId in tests * lintfix * add table for cohort metrics and add dropdown metric selector, add new metrics * add new metrics * undo unwanted changes * fix casing * add superscript 2 for r-squared * update tests to reflect new metrics * lintfix * add feature flag * fix mae * fix mae calculation * first version of new model overview table * get probability distribution box plot to work * add feature flag for new model overview experience and turn it off by default * add highchart heatmap helper class * remove custom styling * add erroneously deleted line back * remove useless constructor * modularize model overview * show outliers in box plot, fix positioning * remove showmetricsummary * refactor heatmap code into a common class * add featureDropdownRef to allow focusing * add only dataset cohort table, update wrapping code * lintfix * lintfix * build error fix * add chart config flyout (in progress) * add chart config flyout (in progress) * address feedback, use finalized color * adjust feature selection to disable options if limit is reached, add axis config buttons * select all via dropdown * lintfix * refactor box plot calculations and rendering * add style file * textured NaN cells, grid labels on y axis * standardize box plots to use fences * fix merge issues * unify box plot tooltip formatting code, fix bar chart sizing issue * small fixes * rearrange feature dropdown * lintfix * remove commented out code * remove box plot tooltip customization * lintfix * add a few unit tests * unit tests for smaller utilities, localization fixes, consistent flyout flow with confirm/cancel buttons * lintfix * fix chart config flyout update * fix test case * rename files for lint * file rename for lint * release rai-core-flask 0.3.0 (#1361) * upgrade python version used with flask CI to fix segfault error (#1363) * release raiwidgets and responsibleai v0.18.0 (#1360) * fix two bugs (#1364) * Add pre-built cohort into adult census notebook (#1243) * [WIP] Add pre-built cohort into adult census notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * erroranalysis version bump in raiwidgets to 0.1.31 (#1245) * Make cohrtData empty list in case no pre-bdefined cohorts are injected (#1247) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add regression test for pre-defined cohorts in raiwidgets (#1249) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * color (#1248) * Add feature importance box & bar chart (#1241) * refactor * build * build * temp * temp * temp * temp * box * cache * e2e * e2e * fix * e2e fix * e2e * fix e2e * widget * widget * fix * widget * e2e * e2e * e2e * test * test * PreBuilt cohorts UX changes (#1242) * Intial SDK implementation cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic validationf for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add serialized version of cohort config to ResponsibleAiDashboard Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more tests cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix broken builds due to pip upgrade which broke pip-tools (#1185) * refactor matrix filter and area state to be private static (#1179) * Change variable name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort data to dashboard e2e Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohorts filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Document various data validation for cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add new interfaces for pre-built cohort Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add prebuilt cohort walking logic in UI and add more data validation scenarios Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add basic data validation checks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add logic to translate the Index cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Remove commented out code Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK validations for Index based cohort filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for validating classification outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add error filter validations and add tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for regression dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add fake cohorts for multi-class classification dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add handling of regression filter Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add support for classification outcome in UI Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for Predicted Y and True Y cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI code to handle prediced Y and true Y for pre-built cohort filters Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohort validation with test data to raiwidgets Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add tests for validating Predicted/True Y cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI support for TrueY/PredictedY for classification Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename cohort_filter_list to cohort_list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Rename UI varibles to match SDK Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix duplicate cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add SDK cohorts to notebook Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add dataset validations and add categorical features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for categorical_features Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add code for translating categorical values Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Move cohort processing to a separate file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort translated function into different small functions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change to lowercase for outcome Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Refactor cohort_list validations and converge pytest common functions into fixtures Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add conftest into raiwidgets tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add validations for cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add cohortData test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix sorted imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * isort fix Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI unit tests for cohort translation Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more checks in UI uni test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add UI tests for regression cohorts Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * REmove notebook change Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix typescript build Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort filter values so that cohort filters non-zero points Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix for empty cohort list Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb (#1195) * Simplify the train pipeline responsibleaidashboard-census-classification-model-debugging.ipynb Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments * Update notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Propagate error strings instead of raising exceptions Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code issues Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Make _cohort.py module a public module (#1253) * Make _cohort.py a public module Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add missing file Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * fix notebook build failures due to pywinpty dependency release failing in python 3.6 (#1257) * fix notebook build failures due to pywinpty dependency release failing in python 3.6 * build pywinpty from conda instead * add lowerbound * fixup * fixup * Add supported models and data types to README.md responsibleai (#1259) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * make getting-started notebook a markdown file showing APIs (#1223) * refactor tabs out of RAI dashboard into a separate component (#1256) * Add individual causal scatter chart (#1258) * temp * refactor * test * style fix * comment * minor fix to url for responsibleai package in setup.py (#1260) * Fix UX e2e tests and address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix eslint Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Reset the number of samples in test dataset Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Change cohort name Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: tongy-msft <91754176+tongyu-microsoft@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> * Add more context to download print in fetch_dataset (#1368) * Add missing desired range to counterfactual chart (#1366) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a summary for disaggregated analysis showing ratio and difference (#1367) * add fairness metric table, update existing tables (order, N/A values, grid axis label lengths) * lintfix * merge fixes * addressed eslint issue for null value by creating a dedicated file, localized N/A, updated tooltip text to be wrapped (+ associated unit tests) * keep feature flag * Update PR template to be more intuitive (#1116) * Updating PR template to make sure there's at least 1 box that can be ticked. * replace checkboxes with GitHub emojis for checkmark and x to prevent GitHub from interpreting it as a task list * update PR template based on feedback * lintfix * Update CODEOWNERS file (#1369) * Use global cohort for disaggregated analysis (#1370) * use global cohort for disaggregated analysis and warn user about it * lintfix * Change procedural loops to functions map() and filter() (#1255) * Change procedural loops to function map() and filter() Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix review comment Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add accuracy as multiclass metric (#1371) * support accuracy for multiclass classification * lintfix * update code owner (#1372) Signed-off-by: Ke Xu <xuke@microsoft.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> * Rename _validate_model_analysis_input_parameters() to _validate_rai_insights_input_parameters (#1373) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Cache and serialize the counterfactual explainer (#1374) * Cache and serialize the counterfactual explainer Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix failing tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add a top-level pivot to model overview (#1375) * show only disaggregated analysis OR only dataset cohorts * lintfix * basec -> basic typo * Add a toggle to model overview to turn heatmap colors on and off (#1376) * add visual display toggle * lintfix * simplify onVisualDisplayToggleChange method * color replacements * fluentui colors * localization, style file * lintfix * Create rai-e2e package for tests shared with AML (#1378) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * index update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * remove files from wid-e2e Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update datashape Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * add pyspark support to tree surrogate model in error analysis (#1251) * Create e2e package for tests shared with AML (#1383) * new buildable folder Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * remove rai-e2e lib Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * add license Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Add feature and metric configuration flyouts to model overview (#1381) * show only disaggregated analysis OR only dataset cohorts * lintfix * basec -> basic typo * feature config * merge fixes * metric config * lintfix * wrap metric descriptions * lintfix * move constant to constants file to avoid circular dependency, lintfix * move inline code into functions * Add probability distribution line chart to model overview (#1382) * line chart for prob distribution * lintfix * fix race condition in tree view update code which can cause root stats to be ignored on selected cohort in RAI dashboard static view (#1386) * String updates for model overview (#1387) * string updates for model overview * lintfix * Handle the loading of causal models more gracefully (#1377) * Handle the loading of causal models more gracefully Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review commensts and fix test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix code review comment Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Some fixes for data explorer, heatmap, counterfactual components (#1389) * add x axis and update heatmap * cf custom point color should be consistent * cf custom point name should be displayed * disable causal treatment whatif in static view (#1391) * fix big data set for counter factual chart (#1393) Signed-off-by: Ke Xu <xuke@microsoft.com> * alignment fixes for causal (#1392) * Fix feature selection state carry-over bug in model overview (#1390) * Provide feature flight support to dashboard, test environment, and Python widget (#1385) * Python part of exposing flights * add feature flighting * fix path for test case * update snapshot * add none constant, contrib guide update, and unit tests for feature_flights param * flake8 * isort * release raiwidgets and responsibleai v0.18.1 (#1413) * Add warning in counterfactual manager when unable to load explainer (#1412) * Add warning in counterfactual manager when unable to load explainer Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix linting Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Bug fixes on 'Set value' not copying over feature values correctly in what if counterfactual panel (#1416) * fix cf set value bug * remove comment * fix error on machines with pyspark installed where passed dataframe is not spark pandas (#1415) * add postbuild branch trigger (#1417) * Fix causal UI strings according to classification/regression tasks (#1419) * Fix causal UI strings according to classification/regression tasks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint error Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix UI test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix description for model overview (#1425) * fix description for model overview * keep new description for new model overview * fix failing to create error report when filter_features is empty list (#1421) * filter out missing values from what if dropdown to prevent explanation dashboard from crashing (#1418) * Remove |Set Value| blurb in case it is not availble in counterfactual panel (#1426) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add y-axis description to counterfactual feature importance chart (#1423) Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> * Add the user class name to causal UI strings (#1422) * Fix causal UI strings according to classification/regression tasks Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix lint error Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix UI test Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add the user class name to causal UI strings Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> * fix math.min / max for array size more than 10^7 (#1427) Signed-off-by: Ke Xu <xuke@microsoft.com> * upgrade pytest and lightgbm to try to fix random pytest segfault test failures (#1424) s * fix flaky notebook causing build failures by adding retry logic (#1431) * Upper bound SciKit-Learn to address freeze in causal (#1432) <!--- Provide a general summary of your changes in the Title above --> ## Description Replaces #1429 to address #1430 . Causal analysis is getting stuck with the latest release of SciKit-Learn. This contains: - Test case which gets stuck with SciKit-Learn 1.1.0 - Upper bound on SciKit-Learn in `requirements.txt` ## Checklist <!--- Make sure to satisfy all the criteria listed below. --> - [x] I have added screenshots above for all UI changes. - [x] Documentation was updated if it was needed. - [x] New tests were added or changes were manually verified. Signed-off-by: Richard Edgar <riedgar@microsoft.com> * fix dependency chart axis updating with incorrect values in explanation dashboard (#1437) * fix codecov and widget test screenshot uploads (#1428) * release raiwidgets and responsibleai v0.18.2 (#1439) * fix (#1441) Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Fix cohort name conflict and not run few tests for AML (#1442) * fix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Few e2e tests changes to accommodate AML static tests (#1445) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Fix locators logic for string features - data explorer and model statistics components (#1446) * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * fix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * update Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * lintfix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * fix Signed-off-by: vinutha karanth <vinutha.karanth@gmail.com> * Add more unittests RAI dashboard input class (#1448) * Add unit tests for ResponsibleAIDashboardInput Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Add more tests Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Fix imports Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> * Address code review comments Signed-off-by: Gaurav Gupta <gaugup@microsoft.com> Co-authored-by: Bo Zhang <71688188+zhb000@users.noreply.github.com> Co-authored-by: Roman Lutz <romanlutz13@gmail.com> Co-authored-by: Ilya Matiach <ilmat@microsoft.com> Co-authored-by: tongy-msft <91754176+tongyu-microsoft@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xuke444 <40614413+xuke444@users.noreply.github.com> Co-authored-by: Vinutha Karanth <vinutha.karanth@gmail.com> Co-authored-by: Richard Edgar <riedgar@microsoft.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix sort by absolute value from local importance chart
Areas changed
npm packages changed:
Python packages changed:
Tests
Screenshots (if appropriate):
Documentation: