Skip to content

Python: Fix some problems in TaintKind useage#3591

Merged
yoff merged 5 commits into
github:masterfrom
RasmusWL:python-taintkind-fixup
Jun 5, 2020
Merged

Python: Fix some problems in TaintKind useage#3591
yoff merged 5 commits into
github:masterfrom
RasmusWL:python-taintkind-fixup

Conversation

@RasmusWL

Copy link
Copy Markdown
Member

See details in commit messages.

RasmusWL added 3 commits May 29, 2020 12:06
If I wanted to use my own TaintKind and not have any interaction with
`UntrustedStringKind` that wouldn't be possible today since these standard http
libraries import it directly. (also, I wouldn't get any sources of my custom
TaintKind from turbogears or bottle). I changed them to use the same pattern of
`ExternalStringKind` as everything else does.
This QL

```codeql
import python
import semmle.python.dataflow.TaintTracking
import semmle.python.security.strings.Untrusted

from CollectionKind ck
where
    ck.(DictKind).getMember() instanceof StringKind
    or
    ck.getMember().(DictKind).getMember() instanceof StringKind
select ck, ck.getAQlClass(), ck.getMember().getAQlClass()
```

generates these 6 results.

```
1	{externally controlled string}          ExternalStringDictKind	UntrustedStringKind
2	{externally controlled string}	        StringDictKind	        UntrustedStringKind
3	[{externally controlled string}]	SequenceKind	        ExternalStringDictKind
4	[{externally controlled string}]	SequenceKind	        StringDictKind
5	{{externally controlled string}}	DictKind	        ExternalStringDictKind
6	{{externally controlled string}}	DictKind	        StringDictKind
```

StringDictKind was only used in *one* place in our library code. As illustrated
above, it pollutes our set of TaintKinds. Effectively, every time we make a
flow-step for dictionaries with tainted strings as values, we do it TWICE --
once for ExternalStringDictKind, and once for StringDictKind... that is just a
waste.
@RasmusWL RasmusWL requested a review from a team as a code owner May 29, 2020 10:08
@RasmusWL RasmusWL requested a review from yoff May 29, 2020 10:08
@RasmusWL

Copy link
Copy Markdown
Member Author

Uhhhh, I actually forgot to highlight a quite important detail 😬

Any import semmle.python.web.HttpRequest would also import UntrustedStringKind (transitively). By removing this feature, we might actually make queries stop giving results, if the query does not explicitly contain import semmle.python.security.strings.Untrusted, since there would be no non-abstract subclass of ExternalStringKind.

I'll make an update in the change-notes, but don't know if we need more than that?

@yoff

yoff commented May 29, 2020

Copy link
Copy Markdown
Contributor

It seems to me that commit 1 is unrelated to commits 2 and 3? The former is standardising some web libraries while the latter are cleaning up our taint tracking labels. Small as they are, I would probably have expected them to inhabit separate PRs.

In commit 1, before and after look alpha-equivalent to me, but I will take your word for it that UntrustedStringKind is more likely to cause conflicts than ExternalStringKind.

@yoff

yoff commented May 29, 2020

Copy link
Copy Markdown
Contributor

Would queries just fail silently without some error?

@RasmusWL

Copy link
Copy Markdown
Member Author

It seems to me that commit 1 is unrelated to commits 2 and 3? The former is standardising some web libraries while the latter are cleaning up our taint tracking labels. Small as they are, I would probably have expected them to inhabit separate PRs.

Yes, I am sometimes lazy and group things together that are related but could be split into multiple PRs.

In commit 1, before and after look alpha-equivalent to me, but I will take your word for it that UntrustedStringKind is more likely to cause conflicts than ExternalStringKind.

If using instanceof UntrustedStringKind, it wouldn't be possible to become a source of ExternalXmlString (or any other custom TaintKind). Does that clear it up? 😊

class ExternalXmlString extends ExternalStringKind {
ExternalXmlString() { this = "external xml encoded object" }
}

Would queries just fail silently without some error?

If by "fail" you mean "not produce any results", then yes

Comment thread python/ql/src/semmle/python/security/strings/Basic.qll Outdated
Co-authored-by: Taus <tausbn@gmail.com>
@RasmusWL RasmusWL requested a review from tausbn June 4, 2020 15:14
@RasmusWL

RasmusWL commented Jun 4, 2020

Copy link
Copy Markdown
Member Author

@yoff did you see my response above? 😊

@yoff

yoff commented Jun 5, 2020

Copy link
Copy Markdown
Contributor

Yes, so it seems to me that the importance of switching from UntrustedStringKind to ExternalStringKind can only be understood in terms of the surrounding programming. That is, the programming discipline for reusing taint tracking definitions is via inheritance and a set of useful libraries has standardised on inheriting from exactly ExternalStringKind. This PR is bringing bottle.ql in line with that programming discipline. Is that correctly understood?

@yoff

yoff commented Jun 5, 2020

Copy link
Copy Markdown
Contributor

Thanks for the out-of-band explanation, happy to merge now.

@yoff yoff merged commit e5480e4 into github:master Jun 5, 2020
@RasmusWL RasmusWL deleted the python-taintkind-fixup branch June 5, 2020 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants