As Alex points out, PloneTestCase is about integration testing. The "official" Plone testing tutorial even says so - http://plone.org/documentation/tutorial/testing.
Quite often, integration tests are more important in Plone. The first tests you write tends to be:
- does the product install?
- did it set up the necessary bits (new workflows, say)?
- can I create an object of this type in the intended location?
It's quite hard to mock up the underlying bits of Zope and Plone in order to properly write unit test for things like Archetypes objects. For declarative GenericSetup stuff, say, it's impossible, and integration tests are the only meaningful way of testing.
I certainly advocate writing simpler tests if at all possible, though. I often find it easier to write doctests in docstrings for adapters and utilities that contain isolated logic. That implies that logic is isolated in the first place, which is another worthy goal.
Finally, I've had good experiences in Java with using a mock objects library (EasyMock, in this case). I don't often see them used in Python, but they can make writing proper unit tests much faster. Having to type out your own mocks can be fairly time consuming and requires, as David said, a deep understanding of the framework. A mock object needs only declare what methods it expects to have called on itself, in what order, with what parameters. Often, that's a more meaningful way of specifying the contract between your component and the mocked one.
Martin
Test only what you write
Posted bytseaverat
2008-05-29 07:45 AM
> As Alex points out, PloneTestCase is about integration testing.
> The "official" Plone testing tutorial even says so -
> http://plone.org/documentation/tutorial/testing.
Try running just the unit tests for *any* stock Plone product:
you will see the PTC / ZTC spew ("Installing product Foo"). The
Plone tests still can't be run as a whole reliably, either.
> Quite often, integration tests are more important in Plone. The
> first tests you write tends to be:
>
> - does the product install?
> - did it set up the necessary bits (new workflows, say)?
> - can I create an object of this type in the intended location?
>
> It's quite hard to mock up the underlying bits of Zope and Plone
> in order to properly write unit test for things like Archetypes
> objects. For declarative GenericSetup stuff, say, it's
> impossible, and integration tests are the only meaningful way of
> testing.
You don't need to test code you don't write: if your "product
installation" is just types, skins, and workflows in a GS
profile, you don't need to automate testing it: that code is
already tested elsewhere.
If you are writing your own GS handlers which parse such files
(which nobody is doing in Ploneland: see my 'plone_gs' product)
then you should test those handlers (note that I skimped on those
tests during my Memorial Day weekend jag). those tests should *not*
require creating a Plone site, however -- they should use minimal
mock objects.
Example of mocking CMF
Posted bynouriat
2008-05-29 09:24 AM
I have a 13 lines example doctests here that creates a mocked CMF site and tests a content type's method that requires both the "portal_properties" tool and the "portal_catalog" to work. It's very simple to mock these. Mocky, the library I wrote to support this, makes it easy.
Look at the last example here: http://danielnouri.org/blog/devel/zope/plone-doctest-in-isolation.html
Hard to do test isolation
Posted byalexgat
2008-05-30 03:29 AM
I really think test in isolation seems great but it depends on bit of codes.
Some bits of code are integration of your code in the framework. I don't think it make sense to test them in isolation, but rather do integration test. That is the case for object creation if you did not modify eg. __init__ on an Archetype content type.
Other bits are logics and you shall unit test them in isolation.
My feeling is that separing logic piece from integration code is the right way to be able to do good test. It's not always easy :-)
Quite often, integration tests are more important in Plone. The first tests you write tends to be:
- does the product install?
- did it set up the necessary bits (new workflows, say)?
- can I create an object of this type in the intended location?
It's quite hard to mock up the underlying bits of Zope and Plone in order to properly write unit test for things like Archetypes objects. For declarative GenericSetup stuff, say, it's impossible, and integration tests are the only meaningful way of testing.
I certainly advocate writing simpler tests if at all possible, though. I often find it easier to write doctests in docstrings for adapters and utilities that contain isolated logic. That implies that logic is isolated in the first place, which is another worthy goal.
Finally, I've had good experiences in Java with using a mock objects library (EasyMock, in this case). I don't often see them used in Python, but they can make writing proper unit tests much faster. Having to type out your own mocks can be fairly time consuming and requires, as David said, a deep understanding of the framework. A mock object needs only declare what methods it expects to have called on itself, in what order, with what parameters. Often, that's a more meaningful way of specifying the contract between your component and the mocked one.
Martin
> The "official" Plone testing tutorial even says so -
> http://plone.org/documentation/tutorial/testing.
Try running just the unit tests for *any* stock Plone product:
you will see the PTC / ZTC spew ("Installing product Foo"). The
Plone tests still can't be run as a whole reliably, either.
> Quite often, integration tests are more important in Plone. The
> first tests you write tends to be:
>
> - does the product install?
> - did it set up the necessary bits (new workflows, say)?
> - can I create an object of this type in the intended location?
>
> It's quite hard to mock up the underlying bits of Zope and Plone
> in order to properly write unit test for things like Archetypes
> objects. For declarative GenericSetup stuff, say, it's
> impossible, and integration tests are the only meaningful way of
> testing.
You don't need to test code you don't write: if your "product
installation" is just types, skins, and workflows in a GS
profile, you don't need to automate testing it: that code is
already tested elsewhere.
If you are writing your own GS handlers which parse such files
(which nobody is doing in Ploneland: see my 'plone_gs' product)
then you should test those handlers (note that I skimped on those
tests during my Memorial Day weekend jag). those tests should *not*
require creating a Plone site, however -- they should use minimal
mock objects.
Look at the last example here: http://danielnouri.org/blog/devel/zope/plone-doctest-in-isolation.html
Some bits of code are integration of your code in the framework. I don't think it make sense to test them in isolation, but rather do integration test. That is the case for object creation if you did not modify eg. __init__ on an Archetype content type.
Other bits are logics and you shall unit test them in isolation.
My feeling is that separing logic piece from integration code is the right way to be able to do good test. It's not always easy :-)