The following troubleshoot list, covers possible issues that may rise while working with Zend Platform and using Tunneling. These issues have been listed by category.
|
Possible Issue |
Recommended Action |
|
Images are not displayed in the Administration User Interface (GUI) when using Apache 2. |
Add "EnableSendfile Off" to your httpd.conf. Note: this is not a Platform issue if the Sendfile support is broken, images of other web pages on the same server will suffer from the same problem and this action will fix the problem for the entire server. |
The primary tool for investigating Job Queue related issues is the log file. This file is by default located in $install_prefix/logs/JobQueue.log. The php.ini directive for defining the log file's location is: zend_monitor.log_dir=.
|
Possible Issue |
Recommended Action |
|
API use and queue functionality |
|
|
Re-queuing job fails repeatedly |
When adding jobs, failure can be caused by various reasons, (e.g. illegal file path, queue is full etc. Look at the log or ZendAPI_Queue::getLastError() for error details). As the failure might be inherent in the job itself, re-queueing the job won't help in some cases. Only re-queue when it could solve the problem (e.g. if queue was full try re-queuing after jobs were removed). |
|
Job that was added to the queue isn't in it. |
Make sure to test the return value of addJob(). addJob() returns false upon failure. Use the ZendAPI_Queue::getLastError() which returns the error string and can help determine if the failure reason is related to the queue. If so correct the value and check again. If the failure is due to high load (for example queue is full or connections get timed out) the relevant JobQueue parameters may be changed (e.g. higher maximum queue depth or longer timeout). Another approach is to try scheduling these jobs to a less busy time. |
|
How can I detect jobs that had some kind of logical failure in them? |
Logical Job failures (such as an invalid credit card number) can be detected and tracked if you use the set_job_failed() API. If this function is called when job is being run, it will show as Logically Failed in the Administration Console in Job Queues | Jobs.. |
|
What happens to jobs that were dependent on a job that failed? |
When a job fails, all it's dependent jobs will remain in state "waiting for predecessor", handling such cases is up to the user (possibly fix the job that failed or run a cleaner job to remove dependent jobs). |
|
What happens to a job that failed logically (e.g. because database was temporarily unavailable) but still has recurrences? |
If a job fails logically, but still has recurrences it will keep recurring as usual. |
|
A job changes to status scheduled although it wasn't scheduled |
When job execution failure occurs, due to the re-queuing mechanism, it's state will immediately change to Scheduled. Job states will get the status "execution failure" only after it failed all retries. Execution retry policy can be configured with the directives: zend_jq.initial_backoff, zend_jq.backoff_factor, zend_jq.max_exec_failures |
|
The job isn't performed at the exact time it was scheduled to |
There are no absolute guarantees on job execution times. Jobs are usually performed very close to their scheduled time, but if the load is very high the Job Queue prioritizes execution to be delayed to prevent overloading the system. |
|
The JobQueue's log shows jobs as complete but the job didn't do anything. |
Jobs may be performed successfully by the JobQueue, but the script performed might have an error, thus it is best to monitor the fastcgi processes that perform the jobs to catch these errors (e.g. by loading a Monitor extension ( no need to load other extensions such as Platform) |
|
The daemon isn't updated immediately after actions from the client. |
JobQueue is an asynchronous system, therefore you need to allow time for the commands from the API and other operations to take place (e.g. need to wait between adding addJob() and getJob() ) |
|
Performance |
|
|
Requests from the client aren't handled quickly by the daemon. |
The "number of requested worker threads" should be adjusted to the incoming connections load. This is usually the number of Apache clients, but if the application has jobs that add jobs the amount should be increased - this also includes the JobQueue's number of fastcgi clients. Note: The "number of request worker threads" can be lower than the max concurrent requests, as connections are queued (currently up to 30 connections are queued, this can't be configured by the user). |
|
Jobs from the daemon aren't handled quickly by the fastcgi processes |
The "number of process workers" should equal the number of max fastcgi processes. A lower number may mean under-utilization of the fastcgi processes. |
|
Is there a limit to the number of jobs that can be stored? |
The JobQueue is reliant on the MySQL database that contains the jobs, and therefore, database capacity etc. must be taken into account to set the amount of Jobs to suit the database's capacity (such as determining history cleanup times etc.). |
|
JobQueue shows low performance |
Make sure your hardware can handle the load and that there are enough:
|
|
General |
|
|
There is no communication between client and daemon |
Communication between the client and daemon is via TCP, make sure it's configured correctly (ports, firewalls etc.) and that the requests reach the daemon. A simple way to verify this is to increase the JobQueue log verbosity level and check for messages about connections received. If there are none, the request hasn't reached the JobQueue daemon. |
The ModCluster PHP extension when associated with a webserver (like Apache) communicates with the SCD (Session Cluster Daemon.)
There are two log files that may be checked for problems:
modcluster.log (associated with the ModCluster extention output)
ZendSessionManager.log (associated with the SCD output)
The modcluster log hardly every shows hard errors, however, the following actions can be tracked:
Connectivity with local SCD
Cookies, received from clients
The level of debug messages in the logs is defined by the following zend.ini directives:
ModCluster - mod_cluster.verbosity_level (0-5) [1] (the log level can be changed only upon restart)
SCD - mod_cluster.daemon.verbosity_level (0-5) [1] (0 - CRITICAL and WARNING, 1 - informative, others - debug. The debug level can be adjusted in runtime (see the section about messenger), log rotation is possible also.
The logs are saved in the location defined by the directive: zend_monitor.log_dir.
On a single session clustering machine, there can be several Modcluster processes (up to hundreds) talking to the SCD. They are forked as parts of the Apache processes. The SCD daemon can run several threads as defined in the zend.ini directive mod_cluster.number_of_threads.
As a general guideline, this number can match the count of "virtual" CPUs installed on the system, so if the system has 1 hyper-threading CPU, the number of threads can be set to 2.
Raising this value higher WILL NOT significantly improve the overall system throughput, since each SCD thread is already designed to support multiple MC clients.
ModCluster communicates with the SCD via UNIX domain (much faster) or TCP sockets on UNIX and TCP sockets on Windows. This is set by the directive: mod_cluster.network.use_unix_sockets=0.
The UNIX domain socket appears as a file in the location, set by the directive: zend_temp_dir. If, for some reason you are unable to use the socket (or using Windows), a TCP port for the local communication is set in the directive: mod_cluster.network.tcp_port_local [23456].
|
Possible Issue |
Recommended Action |
|
SC in cluster is not working |
Check that all Nodes have the same SC version installed (both modcluster and SCD) |
|
SCD cannot be contacted |
Check the mod_cluster.network.unix_socket_permissions (octal) [0600] and change the permissions and perform scd.sh restart. |
|
No connection can be made between SCD's |
Check the mod_cluster.allowed_hosts, and make sure it includes the Node IP of the calling SC daemon. |
|
SCDs are not communicating |
All the appropriate ports must be open at the appropriate levels of the cluster network in order for the cluster to operate properly. Firewalls in particular can block essential ports so that the SCD's will not be able to communicate with one another. The local firewall (on the machine itself) needs to be opened for at least 2 of the 3 used ports (the messenger's TCP port is less essential, but recommended). Any firewall that sits between any 2 SCDs must be configured so that the appropriate ports are open or the SCD's will not be able to communicate. |
|
Logs show that sessions cannot be written to the disk. |
check HD space available, quota, and the following directive: mod_cluster.storage.save_path - it might be full or without permissions for the SCD daemon running under: System user / Apache user. This might be solved by running as root, or giving another location for the Sessions [for instance /tmp/zend_sessions/]. |
|
No backup was found for HA mode sessions |
Check if the broadcast is being received in one of the other nodes. |
|
Possible Issue |
Recommended Action |
|
Received Error Message: Cannot communicate with reporting daemon. Reporting disabled. Please restart httpd to re-enable reporting. |
The connection to the monitoring reporting process was broken. This happens when daemon process is either dead or can not timely respond to events sent to it (either stuck or overloaded). |
|
|
|
The Communication Tunnel includes settings in Zend Platform and Zend Studio. The following lists the possible causes and solutions depending on the origin of the problem.
If Studio is unable to connect to the target server, you will get an error message with the response from the server. The table below describes the most likely causes and recommended actions for successfully establishing a connection with the target server.
|
Possible Issue |
Recommended Action |
|
The server address or the port you entered is incorrect. |
Enter the correct server information in the Tunneling Settings dialog. |
|
HTTP authentication is required. |
Enter authentication information in the Tunneling Settings dialog box; then click the 'Send authentication information' checkbox. |
|
The dummy file content or location on the server is incorrect. |
The dummy file on the server side was changed or does not exist. You will need to insure that the correct dummy file with the correct content is placed in the correct directory on the target server (The correct dummy file is created and located properly as part of the Installation procedure. The problem here is post-installation). |
|
You are not allowed to connect with the server via the Communication Tunnel. |
You must have tunneling permissions in the Zend Platform Allowed Hosts Studio Server | Settings. |
If Platform is unable to communicate, there are a number of possible reasons. The table below describes the likely reasons and suggests possible solutions.
|
Possible Issue |
Recommended Action |
|
Zend Studio is not running. |
Run Studio. |
|
The version of Zend Studio you are using is lower than 4.0.0. |
Please install a newer version, if available. Platform’s interface with Studio requires Studio 4.0 (or higher). |
|
Port for auto detection not the same
|
Check that Studio is listening to the same port as the one to which you are trying to connect. |
|
Some other failure happened in the browser or in the Zend Studio |
Use manual settings and if that doesn't work contact Zend Support |