Some long running tasks, such as deleting content, were patched to instead be performed asynchronously using celery. In a nutshell, when a long lasting task is executed by an editor, the following takes place:
- From the Plone UI the editor clicks to fire an action (Say delete)
- The patched Python code, instead of executing the requested action, will register a task with Celery and return
- The editor is redirected to a wait page that will poll the server every 5 seconds.
- Once the background task finishes, the editor is redirected back to the original location, it was supposed to go on step 2, if the code was not patched.
Most of the changes done to the async product were on the UI, almost nothing in the integration with Celery was changed on how it was implemented, other than adding a few extra values to the registered tasks.
Instead of redirecting back to a wait page, the implementation was refactored to include a viewlet with a banner that will show the number of tasks that are running in the background. With a link that will allow the editor to go to a tasks in progress page, with more information about which tasks are currently running, and what they are doing. This viewlet will poll the server every 5 seconds to know the status of running tasks, whether they are still running, or if they are either completed or failed. The viewlet will show the number of tasks currently being processed, or tasks that failed. Tasks that have successfully completed, are discarded. Tasks that have failed will persist in the banner, and they will show one below the other, until the editor dismisses them with the X to the right of the notification.
Example of 2 currently running tasks and a failed one:
The tasks in progress page will show the currently running tasks, with a brief description, and a link to get more information about a specific task.
As of now, the task details page is intended only to get more information about a task that has failed, in order for an editor to be able to get help from an administrator.
This is an example of how it looks like for a currently running task:
This is how it looks like for a task that has completed:
This is how it looks like for a task that has failed:
As it can be seen, when a task fails, it will show what the task was, what the error was, and at the bottom a full traceback of the error.
The table shown in the folder_contents view, is in fact a full Javascript app. When a task finishes successfully, the view of this table will automatically refresh, instead of reloading the full folder_contents page.
When a task that was in a processing state completes successfully (ie. Celery returns its status as success), there are certain conditions under which the current page will be reloaded:
- If the current page is the Tasks in progress.
- If the current page is the Task details for the given task that has completed.
- If the action was Adding an item, and the person is currently at the folder where this item is being added
- If the action was Renaming an item, and the person is currently at either the content being renamed or its parent
- If the action was Deleting an item, and the person is currently at the folder containing such item
- If the action was Pasting an item, and the person is currently at the folder where the item is being pasted
- If the action was Editing an item, and the person is currently at the item itself being edited, or if the person is at the folderish item containing the item, and the changes included changing the item’s title.
It is worth noting that if the current page is either an Add form or an Edit form, the page will never be refreshed
- Add an item
- Edit an item
- Rename an item
- Delete an item
- Paste an item
- Delete an item
- Paste an item
- Rename an item
Events are fired prior to any of the async tasks being created. You can subscribe to each of them, in order to add custom logic. They are the following:
collective.async.interfaces.IAsyncBeforeAdd
collective.async.interfaces.IAsyncBeforeEdit
collective.async.interfaces.IAsyncBeforePaste
collective.async.interfaces.IAsyncBeforeRename
collective.async.interfaces.IAsyncBeforeDelete
You can subscribe to them in the same way you would subscribe to any other event. As an example consider the following ZCML:
<subscriber for="Products.CMFCore.interfaces.IContentish
collective.async.interfaces.IAsyncBeforeAdd"
handler=".event_subscribers.do_something_before_add" />
- Decide if there is need to patch other long lasting actions (Workflow change, Sharing tab permissions, etc).
- Add a timeout so if a task takes longer than X minutes, assume it failed