In this article by Dmitry Dulepov, we will discuss why planning an extension is important and how to plan an extension. Planning issues related to web development are covered in depth in many specialized books. Here, we will cover planning only with regard to TYPO3 extensions.

Why is Planning Important?

Most open source developers see planning as a boring task. Why plan if one can just go and code? The answer is as simple as the question: The "Go and code" approach does not let us create truly optimal code. Portions of code have to be changed while other portions are written. They often lead to redundant code or uninitialized variables, partially covered conditions, and wrong return results. Code gets a "do not touch" reputation because changing anything renders the whole project unstable. Often the code works, but the project is more a failure than a success because it cannot be extended or re-used.

Another reason for planning is the ease of bug fixing and the costs associated with it. Open source developers often do not think about it until they start selling their services or work to commercial companies.

As shown by recent studies, the cost of problem fixing grows rapidly toward the end of the project. The cost is minimal when development has not started yet, and the person in charge just collects requirements. When requirements are collected and a programmer (or a team of programmers) starts to think how to implement these requirements, a change of requirements, or fixing a problem in the requirements still does not cost much. But it may already be difficult for developers if they came to a certain implementation approach after reviewing requirements. Things become worse at the development stage. Imagine that the selected approach was wrong and it was uncovered close to the end of development. Lots of time is lost, and work may have to start from the beginning. Now imagine what happens if the project is released to the customer and the customer says that the outcome of the project does not work as expected (something was implemented differently (as compared to expectations), or something was not implemented at all). The cost of fixing is likely to be high and overshoot the budget. Next, imagine what would happen if problems occurred when a project went live.

After reading the previous paragraph, some developers may ask how the situation applies to non-commercial development, as there is a false perception that there are no costs associated with it (at least, no direct costs). But, the costs exist! And often they are much more sensitive than financial costs. The cost in non-commercial development is reputation. If a developer's product does not work well or does not work at all or it has obvious flaws, the general opinion about the developer may become bad ("cannot trust his code"). Developers will also have troubles improving because often they do not understand what has gone wrong. But the answer is near. Do not rush! Plan it well! You may even think of something about the future code, and then start coding only when the picture is clear.

Planning is an important part of software development. While freelancers can usually divide their time freely between planning and implementation, many corporate developers often do not have such freedom. And even worse, many managers still do not see planning as a necessary step in software development. This situation is well explained in The parable of the two programmers, which readers of this book are encouraged to read in full.

When it comes to TYPO3, planning is more important than an average application. TYPO3 is very complex, and its implementation is also complex. Without planning, programmers will most likely have to change their already written code to fix unforeseen problems therefore, good planning for TYPO3 extensions is extremely important.

But let us move on and see how to plan an extension.

How to Plan

There are several stages in planning. Typically, each stage answers one or more important questions about development. TYPO3 developers should think about at least three stages:

• Gathering requirements
• Implementation planning
• Documentation planning

Of course, each project is unique and has other stages. But these three stages generally exist in every project.

Gathering Requirements

The first thing that a developer needs to know is what his/her extension will do. While it sounds pretty obvious, not many extension authors know exactly what functionality the extension has in the end. It evolves over time, and often the initial idea is completely different from the final implementation. Predictably, neither the original nor the final is done well.

In the other case, when extension features are collected, though planned and implemented according to plan, they usually fit well together. So, the very first thing to do when creating an extension is to find out what that extension should do. This is called gathering requirements.

For non-commercial extensions, gathering requirements simply means writing down what each extension should do. For example, for a news extension, it may be:

• Show list of news sorted by date
• Show list of latest news
• Show news archive
• Show only a small amount of text in news list view

As we have seen, gathering requirements looks easier than it actually is. The process, however, may become more complex when an extension is developed for an external customer.

Alan Cooper, in his famous About Face book, shows how users, architects, and developers see the same product. From the user's perspective, it looks like a perfect circle. An architect sees something closer to an octagon. A developer creates something that looks like a polygon with many segments connected at different degrees. These differences always exist and each participating party is interested in minimizing them. A developer must not be afraid of asking questions. The cleaner picture he/she has, the better he/she will understand the customer's requirements.

Implementation Planning

When the requirements are gathered, it is necessary to think which blocks an extension will have. It may be blocks responsible for data fetching, presentation, conversion, and so on. In the case of TYPO3 extension implementation, planning should result in a list of Frontend (FE) plugins, Backend (BE) modules, and standalone classes. The purpose of each plug-in, module, and/or class must be clear.

When thinking of FE plugins, caching issues must be taken into account. While most of the output can be cached to improve TYPO3 performance, forms processing should not be cached. Some extensions completely prevent caching of the page when processing forms. But there is a better approach, a separate FE plug-in from the non-cached output.

BE modules must take into account the ease of use. Standard BE navigation is not very flexible, and this must be taken into account when planning BE modules. Certain functionalities can be moved to separate classes. This includes common functions and any public APIs that an extension provides to the other extensions. Hooks or "user functions" are usually placed in separate classes depending on the functional zone or hooked class.

Documentation Planning

A good extension always comes with documentation. Documentation should also be planned. Typically, manuals for extensions are created using standard templates, which have standard sections defined. While this simplifies documentation writing for extension developers, they still have to plan what they will put into these sections.

TYPO3-Specific Planning

There are several planning issues specific to TYPO3. Developers must take care of them before the actual development.

Extension Keys

Each extension must have a unique key. Extension keys can be alphanumeric and contain underscore characters. It may not start with a digit, the letter u, or the test_ prefix. However, not every combination of these symbols makes a good extension key.

An extension key must be descriptive but not too long. Having personal or company prefixes is not forbidden but is not recommended. Underscores should be avoided. Abbreviations should be avoided as well, because they often do not make sense for other users.

Examples of good extension keys are:

• news
• comments
• usertracker
• loginbox

Examples of bad extension keys are:

• news_extension
• mycorp_ustr
• myverygoodextensionthatdoesalotofthings
• mvgetdalot
• john_ext
• div2007

Database Structure

Most TYPO3 extensions use a database to load and/or store their own data. Changing the data structure during application development may seriously slow down development, or may even cause damage to data if some data is already entered into the system. Therefore, it is extremely important to think about an extension's data structure well in advance. Such thinking requires knowledge about how TYPO3 database tables are organized.

Tables in TYPO3 database must have certain structures to be properly managed by TYPO3. If a table does not fulfill TYPO3 requirements, users may see error messages in the BE (especially in the Web | List module), and data may become corrupted.

Every record in every TYPO3 table belongs to a certain page inside TYPO3. TYPO3 has a way to identify which page the record belongs to.

Field Names

TYPO3 requires each table to have two fields with predefined names:

• uid
  This is a unique record identifier. It must be an     auto-incremented integer field.
• pid
  This field identifies which page the record belongs to. If this field is zero, it indicates that the page belongs to the "root level" (Globe icon in the TYPO3 page tree).

There are other fields (optional) whose names can be changed by configuration of the table. But typically, they are kept the same in every table:

• crdate
  Holds a record's creation date and time as Unix time stamp value.
• tstamp
  Holds a record's last modification date and time as Unix time stamp value.
• deleted
  If set to a nonzero value, the record is considered deleted (neither shown in the BE, nor available in the FE). Deleted records stay in the database and can be recovered by certain extensions. If this field is not defined, records are truly deleted from the database.
• hidden
  If set to a nonzero value, the record is hidden (not shown in the FE).
• starttime
  Date and time when a record becomes available (shown in the FE) as Unix time stamp.
• endtime
  Date and time when a record stops being available (shown in the FE) as Unix time stamp.
• cruser_id
  The uid value of the Backend user who created this record. Typically zero, if the record is created by the FE plug-in.
• fe_group
  An FE user can access the record only if the user belongs to one or more groups from this list. Meaning of "access" depends on the application, but usually it means that the record is completely inaccessible to users outside these groups.
• l18n_parent
  Related to localization of records.
• l18n_diffsource
  Related to localization of records.
• sys_language_uid
  Related to localization of records.

Several other fields are optional, but their names are reserved by TYPO3. They are related to workspace and version handling:

• t3ver_oid
• t3ver_id
• t3ver_wsid
• t3ver_label
• t3ver_state
• t3ver_stage
• t3ver_count
• t3ver_tstamp
• t3ver_move_id
• t3_origuid

Other field names are free to use.

Depending on the purpose of the field and its data type inside TYPO3, fields in the database table usually belong to one of the following types:

One question that extension developers often ask is why simple int fields are not recommended for database relations. The answer lies in the way TYPO3 stores references to records. Typically, it is the uid value of another record but it can also have a table name prepended (such as tt_content_10). TYPO3 understands both formats, and the second format makes it possible to use one field to relate to records in different tables. Obviously, the int field may not hold all this information.

When an extension is generated, a database table structure is generated along with other files. Therefore, if extension functions are planned properly, only minimal modifications may be needed during the development process.

Indexes

Indexes are the most successful but tricky aspects of databases. Database indexes help to select data faster. This topic is very large and we can discuss it only at a very basic level.

By default, TYPO3 generates two indexes: one for uid and another for pid columns. The first one selects a record by its unique identifier, while the second speeds up queries for the Web | List module.

Developers should add queries that help their extensions fetch data faster. Here are some tips for creating better indexes.

Many indexes on separate fields do not help. MySQL uses one index at a time. Therefore, if a query consists of many fields, one or more fields should be added to the index. However, if the index is too large, MySQL may choose to ignore it and scan the whole table for the records. Therefore, an index should not consist of more than three to four fields.

Fields in the index should be listed in the same order as they are listed in the query. This helps MySQL choose a proper index.

If a query uses sorting, the sorting field should be included as the last field of the index. If it is not at the end of the index, it is likely that MySQL will ignore the index.

Any text or varchar field should have length specification in the index to minimize the index length.

The EXPLAIN MySQL statement will help a developer identify how indexes are used. It should be used on a real set of data because it uses data to evaluate the query.

Summary

In this article, we have seen how important extension planning is. We reviewed some basic principles of extension planning, saw TYPO3-specific issues, and talked about database structure.

Developer's guide to creating feature rich extensions using the TYPO3 API Covers the complete extension development process from planning and extension generation through development to writing documentation Includes both front-end and back-end development Describes TYPO3 areas not covered in the official documentation (such as using AJAX and eID) Hands on style, lots of examples, and detailed walkthroughs Written by Dmitry Dulepov, TYPO3 core developer and developer of TYPO3 extensions such as RealURL and TemplaVoila http://www.PacktPub.com/typo3-extension-development/book

About the Author

Dmitry Dulepov is a TYPO3 core team member and developer of several popular extensions (such as RealURL, TemplaVoila, comments, ratings, and others). He is known by his active support of the TYPO3 community through TYPO3 mailing lists. In 2008 Dmitry won the contest to appear on the first TYPO3 playing cards. He runs a popular blog where he regularly publishes original tips and articles about various TYPO3 features. In addition to his continuous TYPO3 core and extension development, Dmitry provides support for the TYPO3 translation team on behalf of the core team.