Order Confirmation Random Delivery - EE + PHP7 + wpredis + WP + WooCommerce

v3
#1

In order to improve the performance of a website I built some years ago, I moved the site from an Apache based CentOS + Plesk setup to an nginx based Ubuntu + EasyEngine setup, late last year.

In the beginning there were some struggles with certain functionality on the site, but I was able to resolve almost all issues in the days after the setup. One issue though, has been continuously haunting me since the move to the EasyEngine setup:

When a customer creates an order in the webshop (WooCommerce) the confirmation emails are not always being created or sent. This happens with different types of products, payment methods, coupons etc. Up until today I’ve been unable to pin down where this discrepancy is coming from.

So that’s the problem, now some more insights into the configuration:

  • Server: VPS with 2 Xeon CPU’s, 4GB RAM and 150GB SSD server grade storage.
  • Server Configuration: Ubuntu with latest version of Easy Engine.
  • EasyEngine site configuration: WordPress with PHP7 and wpredis.
  • Caching configuration: Cart, Checkout and Account excluded from redis cache (x-srcache-fetch-status:BYPASS, x-srcache-store-status:BYPASS) through nginx configuration file. Redis Object Cache Plugin in WordPress.
  • Webshop: WooCommerce with Pronamic iDEAL for payments and WooCommerce Print Invoices/Packing Lists for Invoicing purposes.

Troubleshooting:

  1. I’ve verified that non of the relevant WooCommerce pages are being cached by looking for the BYPASS fetch-status and store-status value. Dump of the .conf file underneath.
  2. Initially I ran into this problem while the server was configured to deliver email through and smtp relay service provided by the VPS host. As this relay service did not provide any statistics on (non)successful deliveries, I decided to change to mail delivery to the MailGun API, as this would allow me to exclude the mail server as the problem as well as receive more insights into delivery statistics.
  3. MailGun is displaying very healthy delivery statistics with less than 0.3% of emails being bounced or being marked as undeliverable. MailGun looks specified the usual reasons for misdelivery: mailbox full, mailbox does not exist. At this point I was quite certain that the mail delivery service was not the culprit.
  4. As MailGun is doing it’s job, I decided to start looking into WordPress. Are the mails being created in WordPress and do they get stuck, or do they simply never get made? In order to troubleshoot this, I installed the WP Mail Logging plugin. After being enabled for a couple of weeks I compared WP Mail Logging against logs in MailGun. The results were identical. So it looks like all the mails that get created are successfully delivered to MailGun. Non of the mails in WP Mail Logging display errors.

At this point I started thinking that WooCommerce is unable to successfully create these emails, however it seems to be completely random if or when the emails are or aren’t created, making it very hard to troubleshoot. An extreme example: On May 7th, WP Mail Logging and MailGun display the successful delivery of the new order (admin), new order (client) and your invoice (client) emails for one order. A couple of new account (client) emails were created as well. However, on this day 13 orders were created with the same status (processing). These orders did not trigger the expected new order (admin), new order (client) and your invoice (client) emails.

Any thoughts on further troubleshooting?

Thank you!

Dump of the custom cache exclusion configuration file in conf/nginx/

Don’t serve cached pages when items have been added to cart.

if ( $cookie_woocommerce_items_in_cart = “1” ){

set $skip_cache 1; }

Don’t cache uris containing the following segments.

if ($request_uri ~* “(/cart.|/my-account.|/checkout.|/winkelwagen.|/mijn-account.|/afrekenen.|/winkelwagen/|/mijn-account/|/afrekenen/|/wp-admin/|/xmlrpc.php|sitemap(index)?.xml|[a-z0-9-]±sitemap([0-9]+)?.xml)”) {

set $skip_cache 1; }

#2

Is sendmail installed? EE is not installing it by default. Use this command to verify if its installed (and if its not, just install it and retry): apt-get install sendmail

Alternatively, you can install an SMTP plugin i.e : https://wordpress.org/plugins/wp-mail-smtp/ that will guide all emails created by WooCommerce and WordPress, through an external SMTP server.

#3

Hi ggloveswp,

Thank you for your reply. sendmail is installed and with 1500+ successfully delivered emails in MailGun and WP Mail Logging without any logged errors, I think I can conclude that SMTP is not the issue here, unfortunately.

#4

Your issue sounds very strange. Is there any possibility that receipient server blocks email at all, due to high level of spam indicator triggering?

Are you able to verify that emails are being created via database?

#5

Hi ggloveswp,

If mail delivery fails due to any reason, it is logged by MailGun (That’s the primary reason why I am using MailGun). MailGun is not reporting any weird dropped or bounced emails. The domain is also completely clean: not listed on any blacklist.

I’ll dig around in the database to see if and where WooCommerce saves outbound emails.

#6

This may sound weird but it happened to me. I was migrating a new client from his current webstore to a new woocommerce site and I faced a similar issue. I was prepping a customer’s new test site and every test email I sent was getting my test email accounts with no issues. However the client kept complaining he wasn’t getting his emails when he did his own testing. I dug and dug in the code and through his email account to find really no sign of what was going on. I finally broke down and created an aol.com email account since that is what he had and I tried to recreate the issue and that’s when I found it. Prior to me taking over his web design he had taken all of the email addresses from his current store and email blasted them spam without them ever opting in. Doing this got his domain blacklisted on aol. I went through a painful process of getting his domain whitelisted on aol and finally emails arrived… So just as a rule of thumb I would check out creating a test account on each failed domain to see if they possibly have the domain or server ip blacklisted for what ever reason.

#7

Hi EpicFish,

Thank you for your response.

I’ve actually gone through the process of checking blacklists as well. However this problem seems to be occurring in an earlier part of the chain. As I mentioned, I’m using a plugin in WordPress and MailGun as SMTP service in order to fully monitor all outbound emails. If emails were bouncing or undeliverable, that would show up in MailGun. What I’m seeing is weirder: the emails are simply not being created in WooCommerce/WordPress. Additionally, this is fully at random, as a high number of emails were actually created and delivered.

  • Abram
#8

Facing a similar problem here on a vanilla Ubuntu Server LAMP installation; emails with attachments (PDF invoices) or links (downloadable products), are never passed on to the SMTP plugin, however other transactional emails work. Emails sent from the order page in the backend works as expected. All plugins except payment plugins (NETS/PayPal) and SMTP plugins (Postman SMTP) have been disabled.

No errors are reported in the Apache error or in the woocommmerce log, making us suspect a silent fail occurs somewhere in the sendmail/SMTP call in in the woocommerce core, maybe at the system level? Would be very happy for a pointer on where to start looking for errors here, as no errors are logged in the log files :confused:

#9