Skip to content

/ scikit-learn

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FEA Add Nonnegative LinearRegression #17578

Merged
merged 50 commits into from Aug 6, 2020
Merged

FEA Add Nonnegative LinearRegression #17578

merged 50 commits into from Aug 6, 2020

Conversation

@cmarmo
Copy link
Member

@cmarmo cmarmo commented Jun 12, 2020 •
edited

Reference Issues/PRs

Fixes #8191. Resolves #11076, resolve #8428

What does this implement/fix? Explain your changes.

Adds parameters (positive and maxiter) to
LinearRegression consistent with Lasso and ElasticNet.

Any other comments?

#11076 was already approved by @TomDLT , I think I have addressed @rth comments.

Still missing

  • User Guide entry
  • Example
  • what's new
jknox13 and others added 9 commits May 8, 2018
@jknox13
Fixes #8191. Implements nonnegative LinearRegression through `positiv…
929137c 
…e` parameter.

Similar to PR #8428, but adds parameters (`positive` and `max_iter`) to
`LinearRegression` consistent with `Lasso` and `ElasticNet`. Uses
`scipy.optimize.nnls` under the hood.

**note** `scipy.optimize.nnls` cannot accept a sparse matrix for `X` or
`y`. Passing sparse `X` throws error through `check_X_y`.
@jknox13
Update base.py
a778ed6 
Remove `max_iter` parameter since `scipy.optimize.nnls` does not return `n_iter`.
@jknox13
doctest fix
bd2614b
@jknox13
Implememnts edits suggested by @jnothman.
2ef0b9c
@jknox13
Fix comments raised by @agramfort.
436e23f
@jknox13
Address @agramfort review.
39ff0d1 
Remove `_residues` attribute and unnecessary `np.ravel()`.
@jknox13
Merge branch 'master' into LinearRegression_nnls_feature
cda0263
@cmarmo
Sync with upstream.
fff6b4d
@cmarmo
Address rth comments.
0c1ae9b
@cmarmo
Fix docstrings.
f37fffb
@TomDLT
TomDLT reviewed Jun 12, 2020
View changes
Copy link
Member

@TomDLT TomDLT left a comment

As per my previous comments:

This needs to appear in the user guide. Maybe also add an example, as here?
You will also need to add a mention in doc/whats_new/v0.24.rst.
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/tests/test_base.py Show resolved Hide resolved
@cmarmo
Put new parameter at the end.
51df7dc
@jnothman
jnothman requested changes Jun 13, 2020
View changes
sklearn/linear_model/_base.py Show resolved Hide resolved
cmarmo added 3 commits Jun 14, 2020
@cmarmo
Store residues in single target case.
8b9ba07
@cmarmo
Add optimize.nnls maxiter as input parameter.
6a40d90
@cmarmo
Add test on positivity of coefficients.
3d829e9
@cmarmo cmarmo changed the title [MRG] Nonnegative LinearRegression through positive parameter. [WIP] Nonnegative LinearRegression through positive parameter. Jun 14, 2020
cmarmo added 6 commits Jun 14, 2020
@cmarmo
Rename max_iter to maxiter.
da961d9
@cmarmo
Remove maxiter keywords in nnls().
174de83
@cmarmo
Add example from NelleV PR #8428.
4e31258
@cmarmo
Update cell separator in the example.
6a55fea
@cmarmo
Revert maxiter parameter.
3342da1
@cmarmo
Update what's new and User Guide.
86a1d47
@cmarmo cmarmo changed the title [WIP] Nonnegative LinearRegression through positive parameter. [MRG] Nonnegative LinearRegression through positive parameter. Jun 15, 2020
@NicolasHug
NicolasHug reviewed Jun 15, 2020
View changes
Copy link
Member

@NicolasHug NicolasHug left a comment

thanks @cmarmo , mostly looks good

I think we could also have a test showing that we get different results when positive is True vs False
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/tests/test_base.py Outdated Show resolved Hide resolved
doc/modules/linear_model.rst Outdated Show resolved Hide resolved
doc/modules/linear_model.rst Outdated Show resolved Hide resolved
examples/linear_model/plot_nnls.py Outdated Show resolved Hide resolved
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
cmarmo and others added 4 commits Jun 15, 2020
@cmarmo @NicolasHug
Update examples/linear_model/plot_nnls.py
95dd390 
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
@cmarmo @NicolasHug
Update doc/modules/linear_model.rst
6bca08c 
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
@cmarmo @NicolasHug
Update doc/modules/linear_model.rst
ca05ce6 
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
@cmarmo
Remove description in versionadded.
beafc76
cmarmo added 4 commits Jun 27, 2020
@cmarmo
Sync with upstream.
8f6f59d
@cmarmo
Merge branch 'master' into LinearRegression_nnls_feature
353caab
@cmarmo
Add test for different results for positive=True and posiive=False.
7414b28
@cmarmo
Use y instead of Y in non multioutput tests.
837e756
@NicolasHug
NicolasHug reviewed Jul 5, 2020
View changes
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
doc/modules/linear_model.rst Outdated Show resolved Hide resolved
doc/modules/linear_model.rst Outdated Show resolved Hide resolved
examples/linear_model/plot_nnls.py Outdated Show resolved Hide resolved
sklearn/linear_model/tests/test_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/tests/test_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/tests/test_base.py Show resolved Hide resolved
sklearn/linear_model/tests/test_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/tests/test_base.py Outdated Show resolved Hide resolved
@NicolasHug
NicolasHug reviewed Jul 5, 2020
View changes
Copy link
Member

@NicolasHug NicolasHug left a comment

Thanks @cmarmo , made another pass but mostly looks good
examples/linear_model/plot_nnls.py Outdated Show resolved Hide resolved
doc/modules/linear_model.rst Outdated Show resolved Hide resolved
cmarmo added 7 commits Jul 6, 2020
@cmarmo
Merge branch 'master' into LinearRegression_nnls_feature
2e0c3cd
@cmarmo
Address comments.
03093bc
@cmarmo
Resolve conflicts.
6d61e82
@cmarmo
Remove unused internal reference.
8627b9f
@cmarmo
Merge branch 'master' into LinearRegression_nnls_feature
6ec81fb
@cmarmo
Fix conflicts.
a75e090
@cmarmo
Merge branch 'master' into LinearRegression_nnls_feature
2c11edf
@cmarmo cmarmo requested a review from jnothman Aug 1, 2020
@jnothman
jnothman approved these changes Aug 1, 2020
View changes
Copy link
Member

@jnothman jnothman left a comment

I wouldn't mind a test showing positive=False/True makes no difference if positive=False already gets all non-negative coefficients.

But LGTM.
cmarmo added 2 commits Aug 3, 2020
@cmarmo
Merge branch 'master' into LinearRegression_nnls_feature
a5c110b
@cmarmo
Add test for non-negative problems.
fecc67a
@cmarmo
Copy link
Member Author

@cmarmo cmarmo commented Aug 3, 2020

@jnothman, I have added the test.
Perhaps, @TomDLT or @NicolasHug have some time to check again? Thanks!

@jnothman
jnothman approved these changes Aug 3, 2020
View changes
sklearn/linear_model/tests/test_base.py Show resolved Hide resolved
@NicolasHug
NicolasHug approved these changes Aug 3, 2020
View changes
Copy link
Member

@NicolasHug NicolasHug left a comment

Thanks @cmarmo , minor comments but LGTM
examples/linear_model/plot_nnls.py Show resolved Hide resolved
sklearn/linear_model/tests/test_base.py Outdated Show resolved Hide resolved
sklearn/linear_model/tests/test_base.py Outdated Show resolved Hide resolved
cmarmo and others added 5 commits Aug 3, 2020
@cmarmo
Merge branch 'master' into LinearRegression_nnls_feature
4764f7e
@cmarmo
Address comments.
d9ff696
@cmarmo
Fix comment about sparsity in test.
a33a918
@cmarmo
Merge branch 'master' into LinearRegression_nnls_feature
9f8e12d
@NicolasHug
empty commit to trigger CI
485d21f
@NicolasHug NicolasHug changed the title [MRG] Nonnegative LinearRegression through positive parameter. FEA Add Nonnegative LinearRegression Aug 6, 2020
@NicolasHug NicolasHug merged commit 075d424 into scikit-learn:master Aug 6, 2020
21 checks passed
21 checks passed
@lgtm-com
LGTM analysis: C/C++ No code changes detected
Details
@lgtm-com
LGTM analysis: JavaScript No code changes detected
Details
@lgtm-com
LGTM analysis: Python No new or fixed alerts
Details
ci/circleci: deploy Your tests passed on CircleCI!
Details
ci/circleci: doc Your tests passed on CircleCI!
Details
@circleci-artifacts-redirector
ci/circleci: doc artifact Link to 0/doc/_changed.html
Details
ci/circleci: doc-min-dependencies Your tests passed on CircleCI!
Details
ci/circleci: lint Your tests passed on CircleCI!
Details
@codecov
codecov/patch 100.00% of diff hit (target 98.09%)
Details
@codecov
codecov/project 98.09% (+0.00%) compared to febdd19
Details
@azure-pipelines
scikit-learn.scikit-learn Build #20200806.16 succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (Linting) Linting succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (Linux py36_conda_openblas) Linux py36_conda_openblas succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (Linux py36_ubuntu_atlas) Linux py36_ubuntu_atlas succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (Linux pylatest_pip_openblas_pandas) Linux pylatest_pip_openblas_pandas succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (Linux32 py36_ubuntu_atlas_32bit) Linux32 py36_ubuntu_atlas_32bit succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (Linux_Runs pylatest_conda_mkl) Linux_Runs pylatest_conda_mkl succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (Windows py36_pip_openblas_32bit) Windows py36_pip_openblas_32bit succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (Windows py37_conda_mkl) Windows py37_conda_mkl succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (macOS pylatest_conda_mkl) macOS pylatest_conda_mkl succeeded
Details
@azure-pipelines
scikit-learn.scikit-learn (macOS pylatest_conda_mkl_no_openmp) macOS pylatest_conda_mkl_no_openmp succeeded
Details
@cmarmo cmarmo deleted the cmarmo:LinearRegression_nnls_feature branch Aug 6, 2020
jayzed82 added a commit to jayzed82/scikit-learn that referenced this pull request Oct 22, 2020
@cmarmo @NicolasHug
@jknox13 @jayzed82
FEA Add Nonnegative LinearRegression (scikit-learn#17578)
5e92037 
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Joseph Knox <jknox13@uw.edu>
Co-authored-by: Joseph Knox <joseph.edward.knox@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NicolasHug NicolasHug

@TomDLT TomDLT

@jnothman jnothman

Assignees
No one assigned
Labels
module:linear_model
Projects
None yet
Milestone
No milestone
Linked issues

Successfully merging this pull request may close these issues.

Non-negative least square estimator
5 participants
@cmarmo @jnothman @NicolasHug @TomDLT @jknox13