Skip to content

/ bandit

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add functions susceptible to AST stack overflow to calls blacklist #423

Open
wants to merge 9 commits into
base: master
from
Open

Add functions susceptible to AST stack overflow to calls blacklist #423

wants to merge 9 commits into from

Conversation

@naglis
Copy link

naglis commented Nov 9, 2018

This adds the following calls to the blacklist:

  • ast.literal_eval
  • ast.parse
  • compile
  • dbm.dumb.open

I did not add eval and exec as they are already covered by B307 and B102, respectively, and I'm not sure if duplicating them would make sense.

References:
@naglis
Add functions susceptible to AST stack overflow to calls blacklist
Verified
This commit was signed with a verified signature.
naglis Naglis
GPG key ID: 54DA26AC89685C57 Learn about signing commits
Loading status checks…
afd5cee
@ehooo
Copy link
Contributor

ehooo commented Nov 20, 2018

I think you must create first the Issue.
However, I'm not sure if a DoS on the code must be a rule here.
Maybe @ericwb or @sigmavirus24 could help us to know if this is good new rule or not.
@sigmavirus24
Copy link
Member

sigmavirus24 commented Nov 21, 2018

However, I'm not sure if a DoS on the code must be a rule here.

I'm not sure what you mean by this.

Maybe @ericwb or @sigmavirus24 could help us to know if this is good new rule or not.

I think it's a fair addition to the list of rules. I don't think it will cause too much pain for existing consumers (beyond other PyCQA projects). I do wonder, however, if this is project is starting to be treated less as a list of security vulnerabilities and more as a "Well this could be used in a highly specialized case" stick to hit people with.
@naglis naglis mentioned this pull request Nov 21, 2018
Open
@naglis
Copy link
Author

naglis commented Nov 21, 2018

@ehooo, created issue.

I do wonder, however, if this is project is starting to be treated less as a list of security vulnerabilities and more as a "Well this could be used in a highly specialized case" stick to hit people with.

@sigmavirus24, thanks for feedback. I can't comment on the intended scope of the project. I can see how this rule can introduce potential false positives, but that can be said about most other rules as well.

My biggest issue is with ast.literal_eval which is recommended as a safe(r) alternative to eval by both the official documentation and bandit itself, so I suspect it is commonly[citation needed] used for parsing untrusted data where some evaluation is needed, instead of using eval, which might cause a false sense of security. My goal was to bring attention to that.
@sigmavirus24
sigmavirus24 reviewed Nov 21, 2018
View changes
Copy link
Member

sigmavirus24 left a comment

I see. So what's confusing here is the differences between literal_eval and eval. The former isn't recommended over the latter due to security of parsing but security of execution if I remember correctly. Bandit grew out of OpenStack and OpenStack has large, unwieldy, and complex configuration files for setting up services. Sometimes those services want complex values for an option and resorted to Python data structures in the config. Instead of trying to manually parse them, some used eval but as the official Python docs explain, literal_eval is better:

This can be used for safely evaluating strings containing Python values from untrusted sources without the need to parse the values oneself. It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing.

So it's not so much that one is going to be doing

ast.literal_eval('log.warning(f"{GLOBAL_SECRET}")')

But more

ast.literal_eval('{"foo": "bar", "biz": "baz"}')

If we document "X is bad use Y" and then the user uses Y which causes us to then say "Y is bad, don't use that" we lose all credibility as a tool.

I think what's necessary here is this rule as you've written it, with the other functions included.

I think we also need to update the documentation for the existing eval() and exec() checks to explain that the recommendations are for safer but not safe methods. Everything is a shade of gray of course.

I think the tool should be consistent though. Along similar lines, I think the severity of our ast_overflow check should be Low. Given the typical use-cases I don't think there's significant security impact or severity.
bandit/blacklists/calls.py Outdated
@@ -574,4 +595,18 @@ def gen_blacklist():
'attacks. Consider using tmpfile() instead.'
))

# omitted eval() and exec() as they are already covered by B307 and B102

This comment has been minimized.

Sign in to view
@sigmavirus24

sigmavirus24 Nov 21, 2018

Member

I think we should include them here. It's a different rule.
@ehooo
Copy link
Contributor

ehooo commented Nov 21, 2018

I'm not sure what you mean by this.

@sigmavirus24 I mean that maybe is not importat add rule for denial of services caused by known bug on python
@sigmavirus24
Copy link
Member

sigmavirus24 commented Nov 22, 2018

I mean that maybe is not importat add rule for denial of services caused by known bug on python

@ehooo then should we remove the checks for other standard libraries too? I think we do our best to warn folks about things in the language/standard library enough that this makes sense to add. That's just my 2 pence though. I think I definitely want to hear @ericwb's opinion on this as well at least.
naglis added 2 commits Nov 21, 2018
@naglis
Reduces severity of `ast_overflow` to low
Verified
This commit was signed with a verified signature.
naglis Naglis
GPG key ID: 54DA26AC89685C57 Learn about signing commits
527e647
@naglis
Includes both eval() and exec() to the rule
Verified
This commit was signed with a verified signature.
naglis Naglis
GPG key ID: 54DA26AC89685C57 Learn about signing commits
Loading status checks…
c64829f
@naglis
Copy link
Author

naglis commented Nov 29, 2018

@sigmavirus24 @ehooo,

updated PR to lower rule severity to low, added both eval() and exec() to the rule.

I think we also need to update the documentation for the existing eval() and exec() checks to explain that the recommendations are for safer but not safe methods.

Should this be covered in this PR?
@lukehinds
Merge branch 'master' into ast-overflow
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
392a852
@lukehinds lukehinds requested a review from ericwb Jan 10, 2019
@lukehinds
lukehinds requested changes Jan 10, 2019
View changes
bandit/plugins/ast_overflow_exec.py Outdated
@@ -0,0 +1,45 @@
# -*- coding:utf-8 -*-
#
# Copyright 2018 Hewlett-Packard Development Company, L.P.

This comment has been minimized.

Sign in to view
@lukehinds

lukehinds Jan 10, 2019

Member

Don't need HP in the copyright (unless you work there and did this under HP's time and salary).
naglis and others added 2 commits Jan 14, 2019
@naglis
Removes redundant copyright
Verified
This commit was signed with a verified signature.
naglis Naglis
GPG key ID: 54DA26AC89685C57 Learn about signing commits
Loading status checks…
55d0161
@lukehinds
Merge branch 'master' into ast-overflow
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
c01bc14
@lukehinds
lukehinds approved these changes Jan 14, 2019
View changes
@ghugo
ghugo approved these changes Jan 14, 2019
View changes
@sigmavirus24 sigmavirus24 dismissed their stale review Jan 15, 2019

Haven't re-reviewed but a lot has happened.

ericwb added 3 commits Feb 20, 2019
@ericwb
Merge branch 'master' into ast-overflow
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
30497d1
@ericwb
Merge branch 'master' into ast-overflow
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
d0d6d0b
@ericwb
Merge branch 'master' into ast-overflow
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
547296b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lukehinds lukehinds

@ghugo ghugo

@sigmavirus24 sigmavirus24

@ericwb ericwb

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants
@naglis @ehooo @sigmavirus24 @lukehinds @ghugo @ericwb
You can’t perform that action at this time.