Magento’s Order Management Workflow: Comprehensive but Unrealistic

September 25th, 2009

Here I am, knee deep finalizing a Magento goEmerchant payment module. When done, the module will integrate a Magento store with the goEmerchant XML API which will allow seamless pre-authorizations, captures (on invoicing), refunds, and voids.  All will be handled by the Magento admin panel at the order management pages. Well, I have just discovered how Magento handles things and it is slightly different then what we need it to be. In fact, it is different than what any payment gateway will require. Let me explain:


Assuming that the Magento is configured to Authorize only on checkout, once an order is processed, the payment info will be sent to the gateway and a Credit Card pre-authorization will occur. Resulting in a reference_number which is stored somewhere in the database. In the back-end, an admin will then review all the open orders, or orders under the ‘processing’ status and will create invoices accordingly. When the invoice is created there is an option to Capture the funds at that point. Great! this is almost what needs to happen, but workable. In essence, pre-authorized transactions need to be ‘Settled’ not ‘Captured’ according to goEmerchant’s logic. Not a problem.

What is problematic and will require serious re-plumbing of the Magento logic is the fact that only captured invoices can be voided. Well, this defeats the purpose because that is why we pre-authenticate. In short, according to Magento, a pre-authenticated order can issue an invoice which will then capture the funds and only then one can void the captured, pre-authorized transaction. This is a problem in the logic! What needs to happen, according to goEmerchant, is the ability to void a pre-authorized order without the need to issue an invoice and capture the funds.

I’ll keep you posted on this… to be continued.


Quick Way to Update Fail2ban jail.conf file

September 3rd, 2009
Comments Off

There are plenty of settings in that file, especially if you are running CentOS 5.3 with latest patches and fail2ban from atomic repository. Here is a short list of steps that I follow when setting up new servers:

1. Setup sendmail to start on boot. Make sure it can send emails correctly (Reverse DNS records, hostname config, etc).

2. Make sure that fail2ban starts on boot (I use ntsysv for that).

3. Edit the jail.conf file, type vi /etc/fail2ban/jail.conf

4. Change the time for increased security:

bantime = 86400
findtime = 3600

5. After you save and exit, change all the destination emails to go to root which will then be forwarded to you:

sed -i 's/' /etc/fail2ban/jail.conf

6. Add a forward for all emails to root to your email:

echo "" > ~/.forward

7. Restart fail2ban:

service fail2ban restart

Web Application Hosting ,

Performance Optimized httpd.conf for Magento eCommerce

August 31st, 2009
Comments Off

If you ever run Magento eCommerce on your servers you know that it requires some serious horse power. It is just the nature of the beast, with XML configuration, the usage of the Zend Framework, and MVC design – every page load is not trivial. I realize that there are many lighter and various other forms of web servers, however I always stick to the safest and most reliable which usually happens to be the most popular option out there: httpd. Here is an optimized httpd.conf targeting a dedicated server box for running only httpd (no mysql) and Magento:

A couple of assumptions first:

  • You are running CentOS (I did not test any other distros)
  • You have httpd installed
  • You have lots of RAM (this server has 16GB of RAM)
  • This is a dedicated box
  • You know what you are doing…

Download the file, copy over your httpd.conf, restart httpd:

cp /etc/httpd/conf/httpd.conf ./httpd.conf.old
cat httpd.conf-magento > /etc/httpd/conf/httpd.conf

Once done, you will also need to update the vhosts area of the file near the bottom of it. In case you wanted, here is a direct link to the file:

Web Application Hosting ,

Required PHP Extensions for Magento eCommerce

August 27th, 2009
Comments Off

When I setup a new linux server I make sure to set it up lean since performance is always on my mind. I usually install packages manually, so here is the list of PHP extensions that are needed for the latest stable version of Magento:

  • PHP
  • php-mysql
  • php-mcrypt
  • php-hash
  • php-xml (will be required for Magento 1.4.xx)
  • php-gd
  • php-pdo
  • php-mhash
  • php-soap

Here is the command line that will do the job:

yum install php php-mysql php-mcrypt php-xml php-gd php-pdo php-mhash php-soap

Tip: you should use PECL extension APC. Also known as Alternative PHP Caching. This little extension can speed things up significantly. Assuming you have setup Atomic as a repository in yum, you can run this command:

yum install php-pecl-apc

Web Development

A Quick Way to Download, Install, and Setup pdnsd (local DNS Caching)

August 26th, 2009
Comments Off

Back in December of last year I posted an article about setting up pdnsd. This is a revised version of the same howto, but with a shorter approach for fast setups. I also changed the IP addresses to point to the OpenDNS servers which has proven to be both effective and reliable.

rpm -i pdnsd-1.2.7-par_sl5.x86_64.rpm
echo "server {" > /etc/pdnsd.conf
echo "label=\"opendns\";" >> /etc/pdnsd.conf
echo "ip =,;" >> /etc/pdnsd.conf
echo "}" >> /etc/pdnsd.conf
service pdnsd start

echo START_DAEMON=yes > /etc/default/pdnsd

Edit your resolve.conf file and make sure that the first row has nameserver=, the rest should remain the same. Here is how:

vi /etc/resolv.conf

Restart your network service:

service network restart

Last: make sure that pdnsd auto starts on boot, I use ntsysv for this.

Web Development

Setting noatime and nodiratime for improved disk performance

August 25th, 2009

This is great if you have a server that performs plenty of disk access operations and you are interested in speeding things up a bit. Here is how to do this in three steps. First some assumptions:

  • You have root access
  • You are the only one that will ever need to mount or unmount this file system
  • Running CentOS (this may work, but was not tested on any other linux server)

This is what we are doing:

  1. Editing /etc/fstab to set the noatime and nodiratime flags for the file system
  2. Remounting the drives/file systems (without rebooting)
  3. Checking our work

Step 1: Edit the /etc/fstab, type:

vi /etc/fstab

Add the noatime, nodiratime flags right after the defaults flag in the “/” root mount partition. Repeat to all the partitions that you wish to speed up. Here is how my fstab file looks like after the change (click to enlarge):

etc-fstab-in-vi_with_noatime_nodiratimeStep 2: Remount the file system:

mount -oremount /

Step 3: Check our work:

cat /proc/mounts

Tip: the following command will perform a tiny benchmark test, in case you want to see how fast your hard drives perform. If you want to compare, you should perform a before and after tests.

hdparm -tT /dev/sda

Web Application Hosting, Web Development ,

Howto setup iptables for www and db(mysql) on linux

August 24th, 2009
Comments Off

Since I provide fully managed hosting services to my selective clients, I’ve been setting up iptables more than I can count. After a while I have the tendency to automate common tasks. In the case of setting iptables it can easily be done by downloading and running a shell script. So I created these two shell scripts each one targets a specific server usage: one for WWW servers and the other is for dedicated DB servers.

Here is a couple of assumptions:

  • Using a linux server
  • Has iptables installed and running
  • You have root access to the server (or enough privileges to run these commands)
  • Not a must, but I only tested these scripts on a CentOS server (and many of them…)
  • You know what you are doing!

The scripts simply clears the current settings of your iptables, adds open ports as necessary and restarts the iptables service. In the case of WWW it opens the ports 22, 80, and 443 (SSL). In the case of MYSQL it opens the ports 22, and 3306 and if you wish you can uncomment a line in there to restrict access only to a specific set of IPs. In the case of a dedicated server you may only want the corresponding WWW server to be able to access the MYSQL server. Here are the steps, I will divide them by the type of the server:

Setting iptables on a WWW server:

chmod +x iptables-www

Setting iptables on a MySQL server:

chmod +x iptables-mysql

Note: there is absolutely no warranty that this will work, it is provided with the sole hope that it may save you a few minutes or a couple of hours.

Web Application Hosting ,

CentOS 5.3 Install Essentials

August 23rd, 2009
Comments Off

When I setup a new server, I typically install it with nothing checked in the packages list of the installation process. I like using yum update first and then running yum install on the packages that I absolutely need. Clean and mean is my favorite way to run a Linux server. Two main reason are behind this: one is performance, this is a bit obvious: the less you got on the HD and processes running in the background the faster the server. Two is security: the less software you have installed your vulnerability “surface area” is smaller.

While installing it clean is great, I do have a minimum set of tools that I usually need in any server. Most of these tools are small and do not require background services so I install them almost by default. You should check if they suite your needs and use it at your discretion. Hint: the goal here is to copy and paste once a new server is installed.

(correction) Before I can use the next command I need to install wget:

yum install wget

Add the Atomic repository (newer versions of LAMP, some security packages):

wget -q -O - | sh

Install basic packages:

yum install unzip sendmail ntsysv fail2ban logrotate pdns

LAMP: Linux Apache MySQL PHP, Web Application Hosting ,

Howto change the Base URL in Magento via command line

August 7th, 2009

A quick howto that complements chandansweb’s article on how to change the Base URL using phpMyAdmin. On production servers I do not install phpMyAdmin and therefore I need to use command line. Here is how in 3 steps:

Step 1: Login to mysql (command line) and update two records:
Note that the values in square brackets need to be filled with your specific values.

update [prefix]core_config_data set value="http://[domain]/[rootfolder/]" where path='web/unsecure/base_url';
update [prefix]core_config_data set value="http://[domain]/[rootfolder/]" where path='web/secure/base_url';

Step 2: Clear cache by removing entries in the /var/cache/* folder:
Careful with this one, make sure you are pointing to the right folder.

rm -rf /var/www/[path to html folder]/var/cache/*

Step 3: Force Magento to clear its database cache by running this php file:
Note that I recommend that you copy and paste this code in a file at your Magento root folder. Then point your browser to this file, it will trigger the clearing of various cache objects in Magento.

ob_implicit_flush(true); //Saves having to flush manually
set_time_limit(0); //Set time limit to unlimited, though we shouldnt need to


// clean overall cache

// clear 'refresh_catalog_rewrites':
echo 'Catalog Rewrites was refreshed succesfuly<br>';

//  clear 'clear_images_cache':
echo 'Image cache was cleared succesfuly<br>';

//  clear 'refresh_layered_navigation':
echo 'Layered Navigation Indices was refreshed succesfuly<br>';

That should do it, you should now be able to point to your production store and it will pull the right URLs. It worked for me after various attempts of other howto’s. The above steps are the ones that worked for me.

Magento , ,

Is Magento for eCommerce Scalable?

August 4th, 2009
Comments Off

My recent experience with Magento brings a doubt: is Magento Scalable? I am finalizing a plugin that will allow catalog synchronization with a POS system. Some stores have over 10,000 products, skus to be accurate. I have extended the core classes to accommodate for the product import with certain attributes and was shocked to discover that for each new product saved, or in code terms, each $product->save() it was hammering the DB with approximately 1,000 queries. That also meant that importing 1,000 products took approximately 3-5 minutes on a powerful server which included 15K SCSI hard drives.


Writing directly to the database is not an option because it will potentially introduce a problem with any upgrade efforts and in general is not recommended. There is a limit to how much hardware is being thrown at Magento in order to cover for any performance imperfections. Additionally, almost any interaction with the Magento support team results in lists of mysql configuration best practices which are already in place. Magento: you can run but you cannot hide! Perhaps you cannot run either.

The bottom line is that I am positive that there is a solution, and I am 100% convinced that the Magento developers are working on a solution as I write this blog. It is only in the meantime that it is painful to see all these great sites serving ‘gummy’ pages and developers asking how to add the ‘Please wait while loading’ AJAX loader to the front end of the store…