[Twisted-Python] Python 3 code coverage not working?

[Craig Rodrigues]

Hi,

I submitted this pull request:
https://github.com/twisted/twisted/pull/224

and codecov is saying that only 66.67% of the patch is covered because it
is not hitting the Python 3 specific block:
https://codecov.io/gh/twisted/twisted/compare/502f3a1e2cc125d214b6b7df5b173aa91fc7f966...d286bf22f8b653f75dc3a22cedbe295370435163

This buildbot ran the build on Python 3 and uploaded a coverage report:
https://buildbot.twistedmatrix.com/builders/fedora22-py3.4-coverage/builds/630

Is something wrong here? I don't understand how Python 3 coverage is not
reported.
--
Craig

[Adi Roiban]

Post by Craig Rodrigues
Hi,
https://github.com/twisted/twisted/pull/224
and codecov is saying that only 66.67% of the patch is covered because it
https://codecov.io/gh/twisted/twisted/compare/502f3a1e2cc125d214b6b7df5b173aa91fc7f966...d286bf22f8b653f75dc3a22cedbe295370435163
https://buildbot.twistedmatrix.com/builders/fedora22-py3.4-coverage/builds/630
Is something wrong here? I don't understand how Python 3 coverage is not
reported.

Only the modules (including the test modules) from dist3.py are executed on
Python3
https://github.com/twisted/twisted/blob/trunk/twisted/python/dist3.py

In the header of that file you can see

# -*- test-case-name: twisted.python.test.test_dist3 -*-

and also in the test modules we have

twisted.python.test.test_dist3

---------------

That is you need to make the changes in that module

twisted.python.test.test_dist is not executed on Python3 ... hence its
coverage is not reported on Python3

Hope this helps!

[Craig Rodrigues]

Post by Adi Roiban

Post by Craig Rodrigues
Hi,
https://github.com/twisted/twisted/pull/224
and codecov is saying that only 66.67% of the patch is covered because it
https://codecov.io/gh/twisted/twisted/compare/502f3a1e2cc125d214b6b7df5b173aa91fc7f966...d286bf22f8b653f75dc3a22cedbe295370435163
https://buildbot.twistedmatrix.com/builders/fedora22-py3.4-coverage/builds/630
Is something wrong here? I don't understand how Python 3 coverage is not
reported.

Only the modules (including the test modules) from dist3.py are executed
on Python3
https://github.com/twisted/twisted/blob/trunk/twisted/python/dist3.py
In the header of that file you can see
# -*- test-case-name: twisted.python.test.test_dist3 -*-
and also in the test modules we have
twisted.python.test.test_dist3
---------------
That is you need to make the changes in that module
twisted.python.test.test_dist is not executed on Python3 ... hence its
coverage is not reported on Python3

Is it possible to add the actual invocation of setup.py to coverage?

That seems like a hole that the actual invocation of setup.py that does the
build and install
is not part of the coverage reporting. setup.py is just a Python script,
so why can't it be run under coverage?

--
Craig

[Adi Roiban]

Post by Craig Rodrigues

Post by Adi Roiban

Post by Craig Rodrigues
Hi,
https://github.com/twisted/twisted/pull/224
and codecov is saying that only 66.67% of the patch is covered because
https://codecov.io/gh/twisted/twisted/compare/502f3a1e2cc125d214b6b7df5b173aa91fc7f966...d286bf22f8b653f75dc3a22cedbe295370435163
https://buildbot.twistedmatrix.com/builders/fedora22-py3.4-coverage/builds/630
Is something wrong here? I don't understand how Python 3 coverage is
not reported.

Only the modules (including the test modules) from dist3.py are executed
on Python3
https://github.com/twisted/twisted/blob/trunk/twisted/python/dist3.py
In the header of that file you can see
# -*- test-case-name: twisted.python.test.test_dist3 -*-
and also in the test modules we have
twisted.python.test.test_dist3
---------------
That is you need to make the changes in that module
twisted.python.test.test_dist is not executed on Python3 ... hence its
coverage is not reported on Python3

Is it possible to add the actual invocation of setup.py to coverage?
That seems like a hole that the actual invocation of setup.py that does
the build and install
is not part of the coverage reporting. setup.py is just a Python script,
so why can't it be run under coverage?

We can have a test for setup.py, it just needs someone to write that test :)

Regards,
Adi

[Craig Rodrigues]

Post by Craig Rodrigues
Is it possible to add the actual invocation of setup.py to coverage?
That seems like a hole that the actual invocation of setup.py that does

Post by Craig Rodrigues
the build and install

is not part of the coverage reporting. setup.py is just a Python script,

Post by Craig Rodrigues
so why can't it be run under coverage?

We can have a test for setup.py, it just needs someone to write that test :)

Why can't we run setup.py directly under coverage, during the coverage
build?
Isn't running setup.py during the build one way of testing it?

--
Craig

[Craig Rodrigues]

Post by Adi Roiban
Only the modules (including the test modules) from dist3.py are executed
on Python3
https://github.com/twisted/twisted/blob/trunk/twisted/python/dist3.py
In the header of that file you can see
# -*- test-case-name: twisted.python.test.test_dist3 -*-
and also in the test modules we have
twisted.python.test.test_dist3
---------------
That is you need to make the changes in that module
twisted.python.test.test_dist is not executed on Python3 ... hence its
coverage is not reported on Python3

In https://github.com/twisted/twisted/pull/224 , I made no changes to
twisted/python/dist3.py, so
making changes to test_dist3.py will not help.

I added twisted.python.test.test_dist to the list of tests executed on
Python 3 in https://github.com/twisted/twisted/pull/288 and that
fixed the coverage problem.

I still think that setup.py should be invoked under coverage (I assume in
tox.ini) in the coverage build,
without writing a separate test for setup.py

--
Craig

[Adi Roiban]

Post by Craig Rodrigues

Post by Adi Roiban
Only the modules (including the test modules) from dist3.py are executed
on Python3
https://github.com/twisted/twisted/blob/trunk/twisted/python/dist3.py
In the header of that file you can see
# -*- test-case-name: twisted.python.test.test_dist3 -*-
and also in the test modules we have
twisted.python.test.test_dist3
---------------
That is you need to make the changes in that module
twisted.python.test.test_dist is not executed on Python3 ... hence its
coverage is not reported on Python3

In https://github.com/twisted/twisted/pull/224 , I made no changes to
twisted/python/dist3.py, so
making changes to test_dist3.py will not help.

I added twisted.python.test.test_dist to the list of tests executed on

Post by Craig Rodrigues
Python 3 in https://github.com/twisted/twisted/pull/288 and that
fixed the coverage problem.

I think that dist.py vs dist3.py was made on purpose as 2 different file.
That is, you should not have any `if _PY3` code in any of those files.

But I might be wrong :)