Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing Request: Create unit tests for components and documentation #34

Closed
notadamking opened this issue Oct 12, 2019 · 18 comments
Closed

Testing Request: Create unit tests for components and documentation #34

notadamking opened this issue Oct 12, 2019 · 18 comments

Comments

@notadamking
Copy link
Collaborator

@notadamking notadamking commented Oct 12, 2019

Funding amount:
1.5 ETH (~$300)

Description:
Create fully-featured unit tests for each existing component in /tensortrade and the documentation in /docs.

Requirements:

  • Each component should be tested via pytest in its corresponding /tests/tensortrade/**/test_*.py file.
  • Each use case of each component should be 100% covered via the unit tests.
  • Each function parameter and class argument that affects a component's functionality should be covered by the unit tests.
  • Each documentation feature and page should be tested via pytest in its corresponding /docs/**/test_*.py file.
  • Each test should be passing.
  • Any bugs in components should be fixed during this process.
  • Note: You are not required to implement unimplemented methods that raise a NotImplementedError().

TensorTrade tutorial article

@notadamking notadamking changed the title Testing Request: Test each existing component in the framework Testing Request: Create unit tests for components and documentation Oct 12, 2019
@gitcoinbot
Copy link

@gitcoinbot gitcoinbot commented Oct 12, 2019

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


This issue now has a funding of 0.5 ETH (91.12 USD @ $182.24/ETH) attached to it as part of the tensortrade-org fund.

@gitcoinbot
Copy link

@gitcoinbot gitcoinbot commented Oct 12, 2019

Issue Status: 1. Open 2. Cancelled


Work has been started.

These users each claimed they can complete the work by 2 days ago.
Please review their action plans below:

1) robertalanm has been approved to start work.

I need a minimum of 30 characters so I think that this will work!
2) oolayemi has been approved to start work.

To use phyton unit testing framework "unittest" for unit testing of the software
3) igetgames has been approved to start work.

TensorTrade looks interesting. I'd like to contribute.
4) zoek1 has been approved to start work.

I'm expirenced python developer, i have exprience designing and building testsuites for python code and also i have some ML experience. My plan is start developing test suites in the following order:

  1. base
  2. features
  3. environments
  4. strategies
  5. slippage
  6. trades
  7. actions
  8. exchanges
  9. rewards

Once is donde, i'll start to develop tests for the examples at the documentation.

I'll inform my progress frequently.
5) funlyonions has been approved to start work.

I'll start with a WIP PR and slowly add to it through the night.

Learn more on the Gitcoin Issue Details page.

@gitcoinbot
Copy link

@gitcoinbot gitcoinbot commented Oct 16, 2019

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


The funding of this issue was increased to 1.5 ETH (260.48 USD @ $173.65/ETH) .

@janus
Copy link

@janus janus commented Oct 20, 2019

@notadamking
Is this still open?

@notadamking
Copy link
Collaborator Author

@notadamking notadamking commented Oct 20, 2019

@janus Yes, go ahead and apply on Gitlab if you'd like to take it on.

@janus
Copy link

@janus janus commented Oct 20, 2019

@notadamking
I have applied ... I would even study your code base so that I would be sound enough to take future tasks.

@notadamking notadamking added the $$$$ label Oct 20, 2019
@janus
Copy link

@janus janus commented Oct 24, 2019

@notadamking
It is very unfortunate that I would not continue to work on this issue. I noticed that my machine architecture is not AIX and Tensorflow is compiled for that. My alternative is to use Anaconda which has Tensorflow but I wont be sure of its version. Is there anything you may suggest in this sphere?

@notadamking
Copy link
Collaborator Author

@notadamking notadamking commented Oct 24, 2019

@janus you can use Docker, via the provided Dockerfile and Makefile. Read the instructions in the Readme if you need.

@janus
Copy link

@janus janus commented Oct 25, 2019

@notadamking
I used Docker via Makefile, however I got the below. I am just testing the one that accompanied the repo.

collected 0 items / 4 errors

==================================== ERRORS ====================================
______ ERROR collecting tests/tensortrade/test_component_registration.py _______
ImportError while importing test module '/tests/tensortrade/test_component_registration.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/tensortrade/test_component_registration.py:5: in
import tensortrade.actions as actions
tensortrade/init.py:2: in
from . import environments
tensortrade/environments/init.py:1: in
from .trading_environment import TradingEnvironment
tensortrade/environments/trading_environment.py:19: in
import tensortrade.exchanges as exchanges
tensortrade/exchanges/init.py:6: in
from .instrument_exchange import InstrumentExchange
tensortrade/exchanges/instrument_exchange.py:23: in
from tensortrade.features import FeaturePipeline
tensortrade/features/init.py:4: in
from . import indicators
tensortrade/features/indicators/init.py:2: in
from .talib_indicator import TAlibIndicator
tensortrade/features/indicators/talib_indicator.py:15: in
import talib
E ModuleNotFoundError: No module named 'talib'
_____ ERROR collecting tests/tensortrade/features/test_feature_pipeline.py _____
ImportError while importing test module '/tests/tensortrade/features/test_feature_pipeline.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/tensortrade/features/test_feature_pipeline.py:4: in
import tensortrade.exchanges as exchanges
tensortrade/init.py:2: in
from . import environments
tensortrade/environments/init.py:1: in
from .trading_environment import TradingEnvironment
tensortrade/environments/trading_environment.py:19: in
import tensortrade.exchanges as exchanges
tensortrade/exchanges/init.py:6: in
from .instrument_exchange import InstrumentExchange
tensortrade/exchanges/instrument_exchange.py:23: in
from tensortrade.features import FeaturePipeline
tensortrade/features/init.py:4: in
from . import indicators
tensortrade/features/indicators/init.py:2: in
from .talib_indicator import TAlibIndicator
tensortrade/features/indicators/talib_indicator.py:15: in
import talib
E ModuleNotFoundError: No module named 'talib'
_ ERROR collecting tests/tensortrade/features/stationarity/test_fractional_difference.py _
ImportError while importing test module '/tests/tensortrade/features/stationarity/test_fractional_difference.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/tensortrade/features/stationarity/test_fractional_difference.py:4: in
import tensortrade.exchanges as exchanges
tensortrade/init.py:2: in
from . import environments
tensortrade/environments/init.py:1: in
from .trading_environment import TradingEnvironment
tensortrade/environments/trading_environment.py:19: in
import tensortrade.exchanges as exchanges
tensortrade/exchanges/init.py:6: in
from .instrument_exchange import InstrumentExchange
tensortrade/exchanges/instrument_exchange.py:23: in
from tensortrade.features import FeaturePipeline
tensortrade/features/init.py:4: in
from . import indicators
tensortrade/features/indicators/init.py:2: in
from .talib_indicator import TAlibIndicator
tensortrade/features/indicators/talib_indicator.py:15: in
import talib
E ModuleNotFoundError: No module named 'talib'
_ ERROR collecting tests/tensortrade/rewards/test_risk_adjusted_return_strategy.py _
ImportError while importing test module '/tests/tensortrade/rewards/test_risk_adjusted_return_strategy.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/tensortrade/rewards/test_risk_adjusted_return_strategy.py:5: in
from tensortrade.rewards import RiskAdjustedReturnStrategy
tensortrade/init.py:2: in
from . import environments
tensortrade/environments/init.py:1: in
from .trading_environment import TradingEnvironment
tensortrade/environments/trading_environment.py:19: in
import tensortrade.exchanges as exchanges
tensortrade/exchanges/init.py:6: in
from .instrument_exchange import InstrumentExchange
tensortrade/exchanges/instrument_exchange.py:23: in
from tensortrade.features import FeaturePipeline
tensortrade/features/init.py:4: in
from . import indicators
tensortrade/features/indicators/init.py:2: in
from .talib_indicator import TAlibIndicator
tensortrade/features/indicators/talib_indicator.py:15: in
import talib
E ModuleNotFoundError: No module named 'talib'
!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!
============================== 4 error in 10.57s ===============================
make: *** [Makefile:8: test] Error 2
Makefile:49: recipe for target 'run-tests' failed
make: *** [run-tests] Error 2

I will work towards solving it.

@janus
Copy link

@janus janus commented Oct 26, 2019

If you are struggling to get tensorflow because your box is not AVX Arch, then check out this, https://github.com/davidenunes/tensorflow-wheels

@janus
Copy link

@janus janus commented Oct 26, 2019

@notadamking
I ran your test and I got the below. Seems like data_frame is not stable or am I missing something? Is your result same as the below? If not what should I do?

kaa@kaa-Lenovo-G40-30:~/buildtool/job/amhello/gitcoin/tensortrade$ pytest tests/
============================================================ test session starts =============================================================
platform linux -- Python 3.6.8, pytest-5.1.1, py-1.8.0, pluggy-0.13.0
rootdir: /home/kaa/buildtool/job/amhello/gitcoin/tensortrade
collected 26 items

tests/tensortrade/test_component_registration.py .....s.....ss..ss [ 65%]
tests/tensortrade/features/test_feature_pipeline.py F. [ 73%]
tests/tensortrade/features/stationarity/test_fractional_difference.py FFFF. [ 92%]
tests/tensortrade/rewards/test_risk_adjusted_return_strategy.py .. [100%]

================================================================== FAILURES ==================================================================
_______________________________________________ TestFeaturePipeline.test_incremental_transform _______________________________________________

self = <test_feature_pipeline.TestFeaturePipeline object at 0x7f3bc80098d0>
data_frame = open low high close
0 NaN NaN NaN NaN
1 NaN NaN NaN NaN
exchange = <tensortrade.exchanges.simulated.fbm_exchange.FBMExchange object at 0x7f3bc8009400>

def test_incremental_transform(self, data_frame, exchange):
    difference_all = FractionalDifference(
        difference_order=0.5, inplace=True)

    feature_pipeline = FeaturePipeline(steps=[difference_all])

    transformed_frame = feature_pipeline.transform(data_frame, exchange.generated_space)

    expected_data_frame = pd.DataFrame([{
        'open': -26.20469322,
        'low': -46.15180724,
        'high': 33.63664884,
        'close': 13.68953482,
    },
        {
        'open': 134.53651465,
        'low': 118.24976426,
        'high': 183.39676584,
        'close': 167.11001545,
    }])
  assert np.allclose(expected_data_frame.values, transformed_frame.values)

E assert False
E + where False = <function allclose at 0x7f3be9d72400>(array([[-26.20469322, -46.15180724, 33.63664884, 13.68953482],\n [134.53651465, 118.24976426, 183.39676584, 167.11001545]]), array([[nan, nan, nan, nan],\n [nan, nan, nan, nan]]))
E + where <function allclose at 0x7f3be9d72400> = np.allclose
E + and array([[-26.20469322, -46.15180724, 33.63664884, 13.68953482],\n [134.53651465, 118.24976426, 183.39676584, 167.11001545]]) = open low high close\n0 -26.204693 -46.151807 33.636649 13.689535\n1 134.536515 118.249764 183.396766 167.110015.values
E + and array([[nan, nan, nan, nan],\n [nan, nan, nan, nan]]) = open low high close\n0 NaN NaN NaN NaN\n1 NaN NaN NaN NaN.values

tests/tensortrade/features/test_feature_pipeline.py:57: AssertionError
____________________________________________ TestFractionalDifference.test_incremental_difference ____________________________________________

self = <test_fractional_difference.TestFractionalDifference object at 0x7f3bc8006940>
data_frame = open low high close
0 NaN NaN NaN NaN
1 NaN NaN NaN NaN
exchange = <tensortrade.exchanges.simulated.fbm_exchange.FBMExchange object at 0x7f3bc8006dd8>

def test_incremental_difference(self, data_frame, exchange):
    transformer = FractionalDifference(
        difference_order=0.5, inplace=True)

    transformed_frame = transformer.transform(data_frame, exchange.generated_space)

    expected_data_frame = pd.DataFrame([{
        'open': -26.20469322,
        'low': -46.15180724,
        'high': 33.63664884,
        'close': 13.68953482,
    },
        {
        'open': 134.53651465,
        'low': 118.24976426,
        'high': 183.39676584,
        'close': 167.11001545,
    }])
  assert np.allclose(expected_data_frame.values, transformed_frame.values)

E assert False
E + where False = <function allclose at 0x7f3be9d72400>(array([[-26.20469322, -46.15180724, 33.63664884, 13.68953482],\n [134.53651465, 118.24976426, 183.39676584, 167.11001545]]), array([[nan, nan, nan, nan],\n [nan, nan, nan, nan]]))
E + where <function allclose at 0x7f3be9d72400> = np.allclose
E + and array([[-26.20469322, -46.15180724, 33.63664884, 13.68953482],\n [134.53651465, 118.24976426, 183.39676584, 167.11001545]]) = open low high close\n0 -26.204693 -46.151807 33.636649 13.689535\n1 134.536515 118.249764 183.396766 167.110015.values
E + and array([[nan, nan, nan, nan],\n [nan, nan, nan, nan]]) = open low high close\n0 NaN NaN NaN NaN\n1 NaN NaN NaN NaN.values

tests/tensortrade/features/stationarity/test_fractional_difference.py:55: AssertionError
______________________________________________ TestFractionalDifference.test_difference_inplace ______________________________________________

self = <test_fractional_difference.TestFractionalDifference object at 0x7f3bc81c0dd8>
data_frame = open low high close
0 NaN NaN NaN NaN
1 NaN NaN NaN NaN
exchange = <tensortrade.exchanges.simulated.fbm_exchange.FBMExchange object at 0x7f3bc81c0fd0>

def test_difference_inplace(self, data_frame, exchange):
    transformer = FractionalDifference(
        difference_order=0.5, inplace=True)

    transformed_frame = transformer.transform(data_frame, exchange.generated_space)

    expected_data_frame = pd.DataFrame([{
        'open': -26.20469322,
        'low': -46.15180724,
        'high': 33.63664884,
        'close': 13.68953482,
    },
        {
        'open': 134.53651465,
        'low': 118.24976426,
        'high': 183.39676584,
        'close': 167.11001545,
    }])
  assert np.allclose(expected_data_frame.values, transformed_frame.values)

E assert False
E + where False = <function allclose at 0x7f3be9d72400>(array([[-26.20469322, -46.15180724, 33.63664884, 13.68953482],\n [134.53651465, 118.24976426, 183.39676584, 167.11001545]]), array([[nan, nan, nan, nan],\n [nan, nan, nan, nan]]))
E + where <function allclose at 0x7f3be9d72400> = np.allclose
E + and array([[-26.20469322, -46.15180724, 33.63664884, 13.68953482],\n [134.53651465, 118.24976426, 183.39676584, 167.11001545]]) = open low high close\n0 -26.204693 -46.151807 33.636649 13.689535\n1 134.536515 118.249764 183.396766 167.110015.values
E + and array([[nan, nan, nan, nan],\n [nan, nan, nan, nan]]) = open low high close\n0 NaN NaN NaN NaN\n1 NaN NaN NaN NaN.values

tests/tensortrade/features/stationarity/test_fractional_difference.py:106: AssertionError
____________________________________________ TestFractionalDifference.test_difference_not_inplace ____________________________________________

self = <test_fractional_difference.TestFractionalDifference object at 0x7f3bc8009cf8>
data_frame = open low high close open_diff_0.5 low_diff_0.5 high_diff_0.5 close_diff_0.5
0 100 50 250 200 ...aN NaN NaN
1 200 150 350 300 NaN NaN NaN NaN
exchange = <tensortrade.exchanges.simulated.fbm_exchange.FBMExchange object at 0x7f3bc8009e48>

def test_difference_not_inplace(self, data_frame, exchange):
    transformer = FractionalDifference(
        difference_order=0.5, inplace=False)

    transformed_frame = transformer.transform(data_frame, exchange.generated_space)

    expected_data_frame = pd.DataFrame([{
        'open': 100,
        'low': 50,
        'high': 250,
        'close': 200,
        'open_diff_0.5': -26.20469322,
        'low_diff_0.5': -46.15180724,
        'high_diff_0.5': 33.63664884,
        'close_diff_0.5': 13.68953482,
    },
        {
        'open': 200,
        'low': 150,
        'high': 350,
        'close': 300,
        'open_diff_0.5': 134.53651465,
        'low_diff_0.5': 118.24976426,
        'high_diff_0.5': 183.39676584,
        'close_diff_0.5': 167.11001545,
    }])
  assert np.allclose(expected_data_frame.values, transformed_frame.values)

E assert False
E + where False = <function allclose at 0x7f3be9d72400>(array([[100. , 50. , 250. , 200. ,\n -26.20469322, -46.15180724, 33.63664884, 13....0. , 150. , 350. , 300. ,\n 134.53651465, 118.24976426, 183.39676584, 167.11001545]]), array([[100., 50., 250., 200., nan, nan, nan, nan],\n [200., 150., 350., 300., nan, nan, nan, nan]]))
E + where <function allclose at 0x7f3be9d72400> = np.allclose
E + and array([[100. , 50. , 250. , 200. ,\n -26.20469322, -46.15180724, 33.63664884, 13....0. , 150. , 350. , 300. ,\n 134.53651465, 118.24976426, 183.39676584, 167.11001545]]) = open low high ... low_diff_0.5 high_diff_0.5 close_diff_0.5\n0 100 50 250 ... -46.151807 33.636649 13.689535\n1 200 150 350 ... 118.249764 183.396766 167.110015\n\n[2 rows x 8 columns].values
E + and array([[100., 50., 250., 200., nan, nan, nan, nan],\n [200., 150., 350., 300., nan, nan, nan, nan]]) = open low high ... low_diff_0.5 high_diff_0.5 close_diff_0.5\n0 100 50 250 ... NaN NaN NaN\n1 200 150 350 ... NaN NaN NaN\n\n[2 rows x 8 columns].values

tests/tensortrade/features/stationarity/test_fractional_difference.py:135: AssertionError
____________________________________________ TestFractionalDifference.test_select_correct_columns ____________________________________________

self = <test_fractional_difference.TestFractionalDifference object at 0x7f3bc8253160>
data_frame = open low high close
0 NaN 50 250 NaN
1 NaN 150 350 NaN
exchange = <tensortrade.exchanges.simulated.fbm_exchange.FBMExchange object at 0x7f3bc8253b00>

def test_select_correct_columns(self, data_frame, exchange):
    transformer = FractionalDifference(
        columns=['open', 'close'], difference_order=0.5, inplace=True)

    transformed_frame = transformer.transform(data_frame, exchange.generated_space)

    expected_data_frame = pd.DataFrame([{
        'open': -26.20469322,
        'low': 50,
        'high': 250,
        'close': 13.68953482,
    },
        {
        'open': 134.53651465,
        'low': 150,
        'high': 350,
        'close': 167.11001545,
    }])
  assert np.allclose(expected_data_frame.values, transformed_frame.values)

E assert False
E + where False = <function allclose at 0x7f3be9d72400>(array([[-26.20469322, 50. , 250. , 13.68953482],\n [134.53651465, 150. , 350. , 167.11001545]]), array([[ nan, 50., 250., nan],\n [ nan, 150., 350., nan]]))
E + where <function allclose at 0x7f3be9d72400> = np.allclose
E + and array([[-26.20469322, 50. , 250. , 13.68953482],\n [134.53651465, 150. , 350. , 167.11001545]]) = open low high close\n0 -26.204693 50 250 13.689535\n1 134.536515 150 350 167.110015.values
E + and array([[ nan, 50., 250., nan],\n [ nan, 150., 350., nan]]) = open low high close\n0 NaN 50 250 NaN\n1 NaN 150 350 NaN.values

tests/tensortrade/features/stationarity/test_fractional_difference.py:156: AssertionError
============================================================== warnings summary ==============================================================
/home/kaa/.local/lib/python3.6/site-packages/eth_utils/applicators.py:32
/home/kaa/.local/lib/python3.6/site-packages/eth_utils/applicators.py:32: DeprecationWarning: combine_argument_formatters(formatter1, formatter2)([item1, item2])has been deprecated and will be removed in a subsequent major version release of the eth-utils library. Update your calls to use apply_formatters_to_sequence([formatter1, formatter2], [item1, item2]) instead.
"combine_argument_formatters(formatter1, formatter2)([item1, item2])"

/home/kaa/.local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py:15
/home/kaa/.local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============================================ 5 failed, 16 passed, 5 skipped, 2 warnings in 55.41s ============================================
kaa@kaa-Lenovo-G40-30:~/buildtool/job/amhello/gitcoin/tensortrade$

@zoek1
Copy link

@zoek1 zoek1 commented Nov 15, 2019

@notadamking could i start work on this? i'm happy to learn and build the testsuite for tensortrade

@notadamking
Copy link
Collaborator Author

@notadamking notadamking commented Nov 15, 2019

@zoek1 Yes, please do.

@gitcoinbot
Copy link

@gitcoinbot gitcoinbot commented Nov 19, 2019

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


The funding of this issue was increased to 2.5 ETH (435.12 USD @ $174.05/ETH) .

@zoek1
Copy link

@zoek1 zoek1 commented Nov 20, 2019

@notadamking i'm complimenting the rewards and features tests. I see there are modules already covered, do you have a priority order for modules after i finish the current complementary tests?

@gitcoinbot
Copy link

@gitcoinbot gitcoinbot commented Dec 8, 2019

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


Work for 2.5 ETH (376.69 USD @ $150.67/ETH) has been submitted by:

  1. @funlyonions

@notadamking please take a look at the submitted work:


@gitcoinbot
Copy link

@gitcoinbot gitcoinbot commented Dec 15, 2019

Issue Status: 1. Open 2. Cancelled


The funding of 2.5 ETH (355.88 USD @ $142.35/ETH) attached to this issue has been cancelled by the bounty submitter

@mwbrulhardt
Copy link
Collaborator

@mwbrulhardt mwbrulhardt commented Aug 20, 2020

This is complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.