Converting a UTF-16 CSV to UTF-8 in PHP

The best solution I’ve found to this is to open the file, and re-write it with the new encoding before opening it for CSV reading;

function get_encoding($filename){
    $encoding='';
    $handle = fopen($filename, 'r');
    $bom = fread($handle, 2);
    //	fclose($handle);
    rewind($handle);

    if($bom === chr(0xff).chr(0xfe)  || $bom === chr(0xfe).chr(0xff)){
        // UTF16 Byte Order Mark present
        $encoding = 'UTF-16';
    } else {
        $file_sample = fread($handle, 1000) + 'e'; //read first 1000 bytes
        // + e is a workaround for mb_string bug
        rewind($handle);

        $encoding = mb_detect_encoding($file_sample , 'UTF-8, UTF-7, ASCII, EUC-JP,SJIS, eucJP-win, SJIS-win, JIS, ISO-2022-JP');
    }
    if ($encoding){
        stream_filter_append($handle, 'convert.iconv.'.$encoding.'/UTF-8');
    }
    return $encoding;
}

/**
* Decode UTF-16 encoded strings.
* 
* Can handle both BOM'ed data and un-BOM'ed data. 
* Assumes Big-Endian byte order if no BOM is available.
* 
* @param   string  $str  UTF-16 encoded data to decode.
* @return  string  UTF-8 / ISO encoded data.
* @access  public
* @version 0.1 / 2005-01-19
* @author  Rasmus Andersson {@link http://rasmusandersson.se/}
* @package Groupies
*/
function utf16_decode($str, &$be=null) {
    if (strlen($str) < 2) {
        return $str;
    }
    $c0 = ord($str{0});
    $c1 = ord($str{1});
    $start = 0;
    if ($c0 == 0xFE && $c1 == 0xFF) {
        $be = true;
        $start = 2;
    } else if ($c0 == 0xFF && $c1 == 0xFE) {
        $start = 2;
        $be = false;
    }
    if ($be === null) {
        $be = true;
    }
    $len = strlen($str);
    $newstr = '';
    for ($i = $start; $i < $len; $i += 2) {
        if ($be) {
            $val = ord($str{$i})   << 4;
            $val += ord($str{$i+1});
        } else {
            $val = ord($str{$i+1}) << 4;
            $val += ord($str{$i});
        }
        $newstr .= ($val == 0x228) ? "\n" : chr($val);
    }
    return $newstr;
}


$newFilename = 'path/to/uploaded/file.scv';
$encoding = get_encoding($newFilename);

/* This is the fix for UTF-16 files which come in from iPad */
if($encoding == 'UTF-16'){
    $utf8Contents = utf16_decode(file_get_contents($newFilename));
    $resource = fopen($newFilename, 'w');
    /* Write the file over the uploaded one */
    fwrite($resource, pack("CCC",0xef,0xbb,0xbf));
    fwrite($resource, $utf8Contents);
}


if (($handle = fopen($newFilename, "r")) !== false) {
    while (($data = fgetcsv($handle)) !== false) {
        ...
    }
}

Curling a URL in PHP

Assume the form object contains an array in the form of params

 $fieldString = http_build_query($form->getParams());

//open connection
$ch = curl_init();

//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL, $this->_dotMailerUri);
curl_setopt($ch,CURLOPT_POST, count($form->getParams()));
curl_setopt($ch,CURLOPT_POSTFIELDS, $fieldString);
// Set this so the result isn't added to stdout and gets returned instead
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

//execute post
$result = curl_exec($ch);

//close connection
curl_close($ch);

Calculating UK Bank Holidays in PHP

This is from the following page at PHP.net: http://php.net/manual/en/ref.calendar.php, copied as a reference here to aid my rapidly depleting hippocampus.

<?php

/*
*    Function to calculate which days are British bank holidays (England & Wales) for a given year.
*
*    Created by David Scourfield, 07 August 2006, and released into the public domain.
*    Anybody may use and/or modify this code.
*
*    USAGE:
*
*    array calculateBankHolidays(int $yr)
*
*    ARGUMENTS
*
*    $yr = 4 digit numeric representation of the year (eg 1997).
*
*    RETURN VALUE
*
*    Returns an array of strings where each string is a date of a bank holiday in the format "yyyy-mm-dd".
*
*    See example below
*
*/

function calculateBankHolidays($yr) {

    $bankHols = [];

    // New year's:
    switch ( date("w", strtotime("$yr-01-01 12:00:00")) ) {
        case 6:
            $bankHols[] = "$yr-01-03";
            break;
        case 0:
            $bankHols[] = "$yr-01-02";
            break;
        default:
            $bankHols[] = "$yr-01-01";
    }

    // Good friday:
    $bankHols[] = date("Y-m-d", strtotime( "+".(easter_days($yr) - 2)." days", strtotime("$yr-03-21 12:00:00") ));

    // Easter Monday:
    $bankHols[] = date("Y-m-d", strtotime( "+".(easter_days($yr) + 1)." days", strtotime("$yr-03-21 12:00:00") ));

    // May Day:
    if ($yr == 1995) {
        $bankHols[] = "1995-05-08"; // VE day 50th anniversary year exception
    } else {
        switch (date("w", strtotime("$yr-05-01 12:00:00"))) {
            case 0:
                $bankHols[] = "$yr-05-02";
                break;
            case 1:
                $bankHols[] = "$yr-05-01";
                break;
            case 2:
                $bankHols[] = "$yr-05-07";
                break;
            case 3:
                $bankHols[] = "$yr-05-06";
                break;
            case 4:
                $bankHols[] = "$yr-05-05";
                break;
            case 5:
                $bankHols[] = "$yr-05-04";
                break;
            case 6:
                $bankHols[] = "$yr-05-03";
                break;
        }
    }

    // Whitsun:
    if ($yr == 2002) { // exception year
        $bankHols[] = "2002-06-03";
        $bankHols[] = "2002-06-04";
    } else {
        switch (date("w", strtotime("$yr-05-31 12:00:00"))) {
            case 0:
                $bankHols[] = "$yr-05-25";
                break;
            case 1:
                $bankHols[] = "$yr-05-31";
                break;
            case 2:
                $bankHols[] = "$yr-05-30";
                break;
            case 3:
                $bankHols[] = "$yr-05-29";
                break;
            case 4:
                $bankHols[] = "$yr-05-28";
                break;
            case 5:
                $bankHols[] = "$yr-05-27";
                break;
            case 6:
                $bankHols[] = "$yr-05-26";
                break;
        }
    }

    // Summer Bank Holiday:
    switch (date("w", strtotime("$yr-08-31 12:00:00"))) {
        case 0:
            $bankHols[] = "$yr-08-25";
            break;
        case 1:
            $bankHols[] = "$yr-08-31";
            break;
        case 2:
            $bankHols[] = "$yr-08-30";
            break;
        case 3:
            $bankHols[] = "$yr-08-29";
            break;
        case 4:
            $bankHols[] = "$yr-08-28";
            break;
        case 5:
            $bankHols[] = "$yr-08-27";
            break;
        case 6:
            $bankHols[] = "$yr-08-26";
            break;
    }

    // Christmas:
    switch ( date("w", strtotime("$yr-12-25 12:00:00")) ) {
        case 5:
            $bankHols[] = "$yr-12-25";
            $bankHols[] = "$yr-12-28";
            break;
        case 6:
            $bankHols[] = "$yr-12-27";
            $bankHols[] = "$yr-12-28";
            break;
        case 0:
            $bankHols[] = "$yr-12-26";
            $bankHols[] = "$yr-12-27";
            break;
        default:
            $bankHols[] = "$yr-12-25";
            $bankHols[] = "$yr-12-26";
    }

    return $bankHols;

}

/*
*    EXAMPLE:
*
*/

header("Content-type: text/plain"); 

$bankHolsThisYear = calculateBankHolidays(2007);

print_r($bankHolsThisYear);

?>

Will output this result:

Array
(
    [0] => 2007-01-01
    [1] => 2007-04-06
    [2] => 2007-04-09
    [3] => 2007-05-07
    [4] => 2007-05-28
    [5] => 2007-08-27
    [6] => 2007-12-25
    [7] => 2007-12-26
)

Thanks to David Scourfield!

Magento Cart Items – Getting Child Product Items

This is how it’s done in the Mage_Checkout_Block_Cart_Item_Renderer_Configurable class. It will return the simple product if available otherwise it will return the product of the cart item.

<?php
public function getChildProduct()
{
    if ($option = $this->getItem()->getOptionByCode('simple_product')) {
        return $option->getProduct();
    }
    return $this->getProduct();
}

Adding a Category Attribute

The installer needs to be run as the Mage_Catalog_Model_Resource_Setup class:

<resources>
    <rayware_setup>
        <setup>
            <module>Rayware_Core</module>
            <class>Mage_Catalog_Model_Resource_Setup</class>
        </setup>
    </rayware_setup>
</resources>
$this->removeAttribute('catalog_category', 'brand');

$this->addAttribute('catalog_category', 'brand', array(
    'group'         => 'General Information',
    'input'         => 'select',
    'type'          => 'int',
    'label'         => 'Brand Link',
    'backend'       => 'eav/entity_attribute_backend_array',
    'source'        => 'rayware/attribute_source_brand',
    'visible'       => true,
    'required'      => false,
    'visible_on_front' => true,
    'used_in_product_listing' => true, // Use this for the attribute to be added to the category flat table!
    'global'        => Mage_Catalog_Model_Resource_Eav_Attribute::SCOPE_GLOBAL,
));

Note: It would be nice to be able to reindex after adding the category attribute so that it propagates to the catalog_category_flat_store table, however it would appear that just running the catalog_category_flat indexer is not sufficient from the installer. Unless all indexers are run, which could be a really slow process, the attribute does not appear to be added to the flat table.

Redirecting the Newsletter Success Page

Sometimes, clients would like the newsletter success action to redirect to an actual page rather than just setting a message and redirecting to the referrer (the default action). Here, we override the newsletter controller and allow a CMS page to be set to the success page which allows the client to edit the content and also change the page at their will.

config.xml

<frontend>
		<routers>
			<newsletter>
				<args>
					<modules>
						<VanVault_Skin before="Mage_Newsletter">VanVault_Skin_Newsletter</VanVault_Skin>
					</modules>
				</args>
			</newsletter>
		</routers>
	</frontend>
 

The Controller

<?php
require_once(Mage::getModuleDir('controllers', 'Mage_Newsletter') . DS . 'SubscriberController.php');

class VanVault_Skin_Newsletter_SubscriberController extends Mage_Newsletter_SubscriberController{
	const XML_PATH_CMS_REDIRECT_PAGE = "newsletter/subscription/redirect_page";
	
	public function newAction(){
		parent::newAction();
		
		if($pageIdentifier = Mage::getStoreConfig(self::XML_PATH_CMS_REDIRECT_PAGE)){
			$this->_redirectUrl(Mage::helper('cms/page')->getPageUrl($pageIdentifier));
		}
	}
}

system.xml

<?xml version="1.0"?>
<config>
	<sections>
		<newsletter>
			<groups>
				<subscription>
					<fields>
						<redirect_page translate="label" module="vanvault">
							<label>CMS Redirect Page</label>
                            <show_in_default>1</show_in_default>
                            <show_in_website>1</show_in_website>
                            <show_in_store>1</show_in_store>
							<frontend_type>select</frontend_type>
							<source_model>vanvault/adminhtml_system_config_source_cms_page</source_model>
							<sort_order>200</sort_order>
						</redirect_page>
					</fields>
				</subscription>
			</groups>
		</newsletter>
	</sections>
</config>

The System Config Source Class

We override the standard cms page selector (Mage_Adminhtml_Model_System_Config_Source_Cms_Page) to add a blank option at the start. This allows the default behaviour of the newsletter success action to remain if no option is selected.

<?php
class VanVault_Skin_Model_Adminhtml_System_Config_Source_Cms_Page extends Mage_Adminhtml_Model_System_Config_Source_Cms_Page{
	public function toOptionArray(){
		$options = parent::toOptionArray();
		array_unshift($options, array(
			'value' => '',
			'label' => Mage::helper('vanvault')->__('-- Please Select --')
		));
		
		return $options;
	}
		
}

Email Translations

When working with email translations, they’re taken from the following place:

app/locale/<<lang_COUNTRY>>/template/email/

So, depending on the set locale for the store (set in System>Configuration>General>Locale) depends on where it gets its translation files from. Inspect the dropdown to see the list of language / country combinations.

Debugging MySQL installed through Homebrew on Yosemite

I recently came across an annoying issue where MySQL would not start on Mac OS X Yosemite, exiting with the following:

ERROR 2002 (HY000): Can&#039;t connect to local MySQL server through socket &#039;/tmp/mysql.sock&#039; (2)

Join me on a journey which makes putting a greased ape on a waltzer and getting him to rate the experience seem like an easy task.

Finding the log files

To fix what’s going wrong, we need to know what’s going wrong. Unfortunately, finding log files (for me at least) can often be a bigger job than fixing the issue at hand. My version of MySQL is installed through Brew. After six days of Googling, I eventually found MySQL’s error log here:

tail -f /usr/local/var/mysql/Daves-MacBook-Pro.local.err

Four and a half days trawling through something which makes War and Peace look like a children’s book, I stumbled on a possible cause for the error:

2015-11-12T15:35:05.354808Z 0 [Warning] InnoDB: Ignoring tablespace `defender/wishlist_item_option` because it could not be opened.

Aha! It looks like one of the databases has become corrupted, MySQL can't open it, and it's failing! Great. Let's try moving the database elsewhere and see how we get on.

ERROR 2002 (HY000): Can&#039;t connect to local MySQL server through socket &#039;/tmp/mysql.sock&#039; (2)

No dice. Right, looking through the log shows up another error:

2015-11-12T15:48:13.051616Z 0 [ERROR] unknown variable &#039;key_buffer=32M&#039;

Variable variables

So, it looks like the variable key_buffer has been replaced with key_buffer_size in a newer version of MySQL. This exists in my MySQL config file:

/usr/local/etc/my.cnf

Changing this sorted my headache, and allowed me to get back to those yummy scrummy databases once more.