You mention that some of the tests take a /long/ time to run, do they also take a long time on travis?
You can look that up in the test logs. At the end of the test run, it lists the most long running tests, split into translation, compilation, execution times.

After a long break which i was unable to work on much, finally got back to it. The issue is that we dont get the actual architecture to run as github actions only provides amd64 runners. Something we could do is have most pipelines run on github actions (all amd64) and have other architectures run on travis.

I'm going to have a look to see if I can find another service that provides a free and fast service with builds running concurrently.

Any thoughts or ideas are welcome :)

Something we could do is have most pipelines run on github actions (all amd64) and have other architectures run on travis.

I think this might be the simplest solution for the time being.

My impression is that the non-amd64 architectures are nice but not essential. I actually wonder if a 32-bit x86 test might be more useful (since that has broken in recent memory without being caught by the test-suite). But I definitely don't think that needs to be part of this change.

Currently trying to figure out if its actually my fault why they take so long to run. Taking this build as an example as an example, the different architectures take 15-30mins to complete, but they seem to be taking too long on my runs.

Edit: taking a look, might be ccache as my runs don't seem to find it (travis ci probably uses the hash of the travis ci file to store it)

@scoder Even with using pythran==0.9.5, cpp fails all the time: https://github.com/davfsa/cython/runs/2260082383 (due to different bugs)

due to different bugs

What I see is that pip and setuptools ended support for Py<=3.5. Meaning, we need to install older pip/setuptools versions for these.

Secondly, I didn't mean to use Pythran 0.9.5 in general but only on Py2. However, if that makes things easier to get working, I'd be ok with only installing Pythran (0.9.7) on Py3 and not at all on Py2.

due to different bugs

What I see is that pip and setuptools ended support for Py<=3.5. Meaning, we need to install older pip/setuptools versions for these.

Secondly, I didn't mean to use Pythran 0.9.5 in general but only on Py2. However, if that makes things easier to get working, I'd be ok with only installing Pythran (0.9.7) on Py3 and not at all on Py2.

I tried also using Pythran 0.9.7 for Py3 and Pythran 0.9.5 for Py2 but it didnt seem to work either.

Commit: davfsa@056bafd
Actions: https://github.com/davfsa/cython/actions/runs/714221503

@scoder Even with using pythran==0.9.5, cpp fails all the time: https://github.com/davfsa/cython/runs/2260082383 (due to different bugs)

I have a question - is pythran really essential for our testsuit?

I have a question - is pythran really essential for our testsuit?

Cython has an optional feature where it can generate Pythran code to try to speed up array operations. So it is necessary to have it to test that.

I wonder if we should switch in two steps:

1. Enable github actions relatively soon (maybe after the next release?)
2. Disable Travis a bit later (after ~1 month).

The advantage of doing it like that is that this PR wouldn't need to be in perfect working order - we could merge it with some tests failing (because we still have Travis), and it'd hopefully be easier to fix things gradually without it all having to go through @davfsa.

Please don't change this PR to do that now because I've suggested this, but I wonder if that might be easier?

Cython has an optional feature where it can generate Pythran code to try to speed up array operations. So it is necessary to have it to test that.

I wonder if we should switch in two steps:

1. Enable github actions relatively soon (maybe after the next release?)

2. Disable Travis a bit later (after ~1 month).

The advantage of doing it like that is that this PR wouldn't need to be in perfect working order - we could merge it with some tests failing (because we still have Travis), and it'd hopefully be easier to fix things gradually without it all having to go through @davfsa.

Please don't change this PR to do that now because I've suggested this, but I wonder if that might be easier?

I would like to get this working perfectly before just cause I'm that kind of person, lol, but if you think its best to do that, I would be fine with it. I would keep trying to fix the issue in my personal github repo and open a pr to fix it (assuming nobody has done it yet). I believe this pythran issue should be the last one before this pr being production ready.

I wonder if we should switch in two steps:

I think there are two parallel things needed. This PR should not remove any CI runs from the travis yml file, and a repo owner should enable github actions for the repo so we can see that the PR works here as it does in @davfsa 's fork.

I tried also using Pythran 0.9.7 for Py3 and Pythran 0.9.5 for Py2 but it didnt seem to work either.

Looking at the current master branch, it looks like for most python versions Travis isn't finding a suitable version of Pythran and it just skipping the tests. So for now that's probably the best solution

Fixed @scoder. I'm fine with merging this now. I could mark the python 3.10-dev tests as allowed_failure or just have them fail as a reminder to fix them 😛.

I have seen how some other packages emulate the different architectures on Github Actions using docker, and they seem to be faster than what I was trying, so you might see another pr finally switching off Travis CI fully. I haven't had time to play around with this and I'm not sure if I will run into the same issues as I did with what I was using, so no promises!

I could mark the python 3.10-dev tests as allowed_failure

That's the thing - it doesn't currently fail on travis. But then, travis has 3.10 alpha 5 and GA has alpha 7. That might make the difference here – but I don't know. So maybe there is an issue on our side and travis just can't tell us yet.

Let's mark it as allowed failure, wait for 3.10 beta to look into it, and get this merged.

I could mark the python 3.10-dev tests as allowed_failure

That's the thing - it doesn't currently fail on travis. But then, travis has 3.10 alpha 5 and GA has alpha 7. That might make the difference here – but I don't know. So maybe there is an issue on our side and travis just can't tell us yet.

Let's mark it as allowed failure, wait for 3.10 beta to look into it, and get this merged.

Just so you know, 3.10 beta released yesterday. For what I have seen in my personal projects, it is only available on linux and macos. I'm going to re-trigger the pipelines so you can see it

3.10 beta released yesterday

Erm, what? It's scheduled for May 3rd.

3.10 beta released yesterday

Erm, what? It's scheduled for May 3rd.

Interesting. Well, the CI seems to say it is using CPython 3.10 beta 1

Release: https://github.com/actions/python-versions/releases/tag/3.10.0-beta.1-105026
PR: actions/python-versions#92

Looking more into this, I found: actions/setup-python#207
It seems to actually be CPython 3.10 alpha 7 but bound to another commit.

I could force it to alpha 7 instead.

I could force it to alpha 7 instead.

No, this is CI, we should use whatever latest build they have.

Thanks a bunch. This is going to help a lot speeding up the development. I can't wait to … stop waiting for travis.

Would it make sense to extend this to windows builds as well to consolidate things? Right now windows is on appveyor and it seems some of the failures go unnoticed. Windows on github actions can use bash shell commands, I don't know if that help since much of Tools/ci-run.sh has to do with apt and gcc.

Would it make sense to extend this to windows builds as well to consolidate things? Right now windows is on appveyor and it seems some of the failures go unnoticed.

I was wondering that too. It's definitely worth a try. I don't think anyone's particularly attached to Appveyor so if it can be made to work on Github Actions then that's probably an improvement.