A reminder about catalog-indexes

We often create new indexes to make content searchable in Plone. Many developers still do it wrong.

Most new sites we create contain facetted search based on eea.facetednavigation. Thus we often create new indexes to make content searchable. One example is www.idea-frankfurt.eu where attributes and relations of projects are used to make them searchable in a useful way.

Almost everything that is to be said about indexes can be found in http://docs.plone.org/develop/plone/searching_and_indexing/indexing.html

But there was a known pitfall when registering indexes in catalog.xml that was only fixed in Plone 4.3.2. Even though Maurits van Rees warned about this in a blogpost which is also reference in the documentation I often see developers making that mistake.

Unless working with Plone 4.3.2+, you should never register a index in catalog.xml because the index will be purged when reinstalling your package. Instead register new indexes in your setuphandlers.py. This was fixed in GenericSetup 1.7.4.

I firmly believe that addons have to be reinstallable without ruining a site.

I extended the method written by Maurits to separate the name of the index and the indexed method (useful when dealing with old code):

# -*- coding: UTF-8 -*-
import logging
from Products.CMFCore.utils import getToolByName
PROFILE_ID = 'profile-my.package:default'


def setupVarious(context):

    # Ordinarily, GenericSetup handlers check for the existence of XML files.
    # Here, we are not parsing an XML file, but we use this text file as a
    # flag to check that we actually meant for this import step to be run.
    # The file is found in profiles/default.

    if context.readDataFile('bgp.webcode_various.txt') is None:
        return

    # Add additional setup code here
    logger = context.getLogger('my.package')
    site = context.getSite()
    add_catalog_indexes(site, logger)


def add_catalog_indexes(context, logger=None):
    """Method to add our wanted indexes to the portal_catalog.

    @parameters:

    When called from the import_various method below, 'context' is
    the plone site and 'logger' is the portal_setup logger.  But
    this method can also be used as upgrade step, in which case
    'context' will be portal_setup and 'logger' will be None.
    """
    if logger is None:
        # Called as upgrade step: define our own logger.
        logger = logging.getLogger('my.package')

    # Run the catalog.xml step as that may have defined new metadata
    # columns.  We could instead add <depends name="catalog"/> to
    # the registration of our import step in zcml, but doing it in
    # code makes this method usable as upgrade step as well.  Note that
    # this silently does nothing when there is no catalog.xml, so it
    # is quite safe.
    setup = getToolByName(context, 'portal_setup')
    setup.runImportStepFromProfile(PROFILE_ID, 'catalog')

    catalog = getToolByName(context, 'portal_catalog')
    indexes = catalog.indexes()
    # Specify the indexes you want, with
    # ('index_name', 'index_type', 'indexed_attribute')
    wanted = (('myindex', 'FieldIndex', 'getMyAttribute'),
              )
    indexables = []
    for name, meta_type, attribute in wanted:
        if name not in indexes:
            if attribute:
                extra = {'indexed_attrs': attribute}
                catalog.addIndex(name, meta_type, extra=extra)
            else:
                catalog.addIndex(name, meta_type)
            indexables.append(name)
            if not attribute:
                attribute = name
            logger.info("Added %s '%s' for attribute '%s'.", meta_type, name, extra)
    if len(indexables) > 0:
        logger.info("Indexing new indexes %s.", ', '.join(indexables))
        catalog.manage_reindexIndex(ids=indexables)

By the way: Besides many other amazing features the package ftw.upgrade also has methods catalog_rebuild_index, catalog_add_index and catalog_remove_index you an use in your upgrade-steps.