Blog

Getting Intel AX200 Wi-Fi to work on Ubuntu 20.04

The information in this post is based on Ubuntu 20.04 with Linux kernel 5.4.0-74-generic. It may or may not apply to other versions.

I recently moved to a new house and finally decided to give moving away from my old cabled gigabit Ethernet to an all-wireless setup a shot – both because of my own laziness as I didn’t really want to drag cables all over the house, but also because Wi-Fi 6 (on paper at least) is capable of even higher speeds than gigabit Ethernet.

I therefore invested in a couple of Wi-Fi 6 adapters for my desktop Ubuntu computers. Believing I had researched well, I opted for the TP-Link AX3000 Archer TX50E PCIe adapter, which is based on the Intel AX200 chip, as many online sources seemed to indicate that this chip is well supported on Linux since kernel 5.1. To be fair, TP-Link only officially supports it on Windows 10.

I have previously had a lot of issues with getting Wi-Fi to work properly on Ubuntu; I don’t know why, but it seems like it’s usually to do with power management that makes the connection unstable, but it seems to me to have been an issue for many years and with many different Wi-Fi chips (perhaps especially those used in USB adapters), so I don’t really know for sure what the problem is.

Anyway, I installed my new fancy PCIe adapter in my main desktop machine, which is currently running Ubuntu 20.04 with kernel 5.4.0-74-generic. The adapter was detected and appeared to be working right away, but dropped the connection in less than a minute. After that I could only get it to work by rebooting and then I had a minute or so of uptime.

I tried many different things; it seemed like the computer loaded the firmware for the AX201 chip instead of the AX200 chip, but even after fixing that, the adapter still wouldn’t work. I disabled power management to no effect.

After countless hours of messing with something that should have just worked out of the box, I finally stumbled upon the only thing that worked and worked well and really fast: install the backport driver from Canonical’s hardware enablement team.

This is really easy and fast to do, so for future reference:

First, add the repository:

sudo add-apt-repository ppa:canonical-hwe-team/backport-iwlwifi

And then install the package:

sudo apt-get install backport-iwlwifi-dkms

Reboot, and Wi-Fi just works like it should 🙂

Notice: I don’t dual boot any of my machines, but if you do dual boot Ubuntu and Windows 10 and you have issues with your Wi-Fi in Ubuntu, it may be due to “fast boot” being enabled in Windows 10, as that option causes Windows to not shut down and release the hardware completely, making Ubuntu unable to claim the Wi-Fi adapter when it needs it. There are many online sources describing this problem, but the gist of it is: disable fast boot in Windows if you dual boot with Ubuntu 🙂

Identifying document class and query for AIF service in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

When asked to add a new field, change some logic or do some other modification to a document service running on an AIF integration port (inbound or outbound), I often end up spending way too much time searching for the actual classes behind the service.

Sure, most of the elements from the standard application are named fairly consistently, but way too often, custom code is not. The forms for setting up document services in AIF does not provide an easy way to see what code actually handles a service, but knowing the name of the service itself (which you can easily find in the port configuration), you can run this small job to save yourself some time – here it’s running for the CustCustomerService, but you can change that to whichever service you need:

static void AIFServiceCheck(Args _args)
{
    AxdWizardParameters param;
    ;

    param   =   AifServiceClassGenerator::getServiceParameters(
                    classStr(CustCustomerService));

    info(strFmt("Service class: %1", param.parmAifServiceClassName()));
    info(strFmt("Entity class: %1", param.parmAifEntityClassName()));
    info(strFmt("Document class: %1", param.parmName()));
    info(strFmt("Query: %1", param.parmQueryName()));
}

Calling AIF document services directly from X++ in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

I was recently helping a customer implement an Application Integration Framework (AIF) inbound port for creating customers based on data they were receiving from another system. As AX already provides the CustCustomer document service, which implements the logic for this, we decided to keep it simple and use the standard solution.

However, it soon turned out that there were a lot of problems getting the external system to generate XML that AX would accept. The XML schema generated by AX is a quite complex one and it also appears there are few bugs in AX that sometimes causes it to reject XML that is schema-valid according to other tools, so all in all, it proved to be less simple than I’d thought.

During the endeavor, I often struggled to figure out what exactly the problem was with certain XML files because the error messages provided by AIF are less than informative. It was also tedious, because I had to wait for the external system to send a new message over MSMQ and then again for AIF to pick up the message and process it before I could see an error.

I therefore investigated whether it’s possible to call the service code directly with a local XML file for somewhat speedier testing and it turns out that it is – and not only that, it’s really simple to do and actually provides a lot more meaningful error messages.

The example job below reads a local XML file and tries to use it with the AxdCustomer class (which is the document class used by the CustCustomer service) to create a customer. You can make similar jobs for all the other document classes, for example AxdSalesOrder, if you need.

static void CustomerCreate(Args _args)
{
    FileNameOpen fileName    = @'C:\TestCustomerCreate.xml';

    AxdCustomer  customer;
    AifEntityKey key;

    #File
    ;

    new FileIoPermission(fileName, #IO_Read).assert();

    customer = new AxdCustomer();

    key = customer.create(  XmlDocument::newFile(fileName).xml(),
                            new AifEndpointActionPolicyInfo(),
                            new AifConstraintList());

    CodeAccessPermission::revertAssert();

    info('Done');
}

The AifEntityKey object returned by the customer.create() method (which corresponds to the “create” service operation in AIF) contains information about which customer was created, among other things the RecId of the created CustTable record.

If what you’re trying to test is an Outbound port instead or if you just need an example of how the XML should look like on the Inbound port, you can also use the document class to export a customer to a file instead by calling the read() method (corresponding to the “read” service operation) instead, like so:

static void CustomerRead(Args _args)
{
    FileNameSave    fileName = @'C:\TestCustomerRead.xml';
    Map             map      = new Map( Types::Integer,
                                        Types::Container);

    AxdCustomer     customer;
    AifEntityKey    key;
    XMLDocument     xmlDoc;
    XML             xml;
    AifPropertyBag  bag;

    #File
    ;

    map.insert(fieldNum(CustTable, AccountNum), ['123456']);

    key = new AifEntityKey();
    key.parmTableId(tableNum(CustTable));
    key.parmKeyDataMap(map);

    customer = new AxdCustomer();
    xml = customer.read(key,
                        null,
                        new AifEndpointActionPolicyInfo(),
                        new AifConstraintList(),
                        bag); 

    new FileIoPermission(fileName, #IO_Write).assert();
    xmlDoc = XmlDocument::newXml(xml);
    xmlDoc.save(fileName);

    CodeAccessPermission::revertAssert();

    info('Done');
}

You should of course replace ‘123456’ with the account number of the customer you wish to read.

Delete a legal entity (company accounts) in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

Notice: There is a very real risk of data loss if you follow the instructions in this post. In fact, it is exactly about deleting data. You should generally not delete legal entities in production environments, only in test or development environments. Usage of this information is at your own risk.

I was recently tasked with completely removing a legal entity (also know as company accounts or data area) from a Dynamics AX 2012 environment. The reason the user didn’t just do it herself from the Legal entities form was that it spewed out some ugly errors about not being able to delete records in certain tables.

After looking into it, I discovered that you cannot delete a legal entity that has transactions. That makes sense, so the obvious solution would be to remove the transactions first, and then delete the legal entity.

Fortunately, Dynamics AX provides a class for removing the transactions of a legal entity, so this is fairly straightforward – although, quite time consuming if you have a lot of data.

The procedure is:

    1. Open the AOT and find the class SysDatabaseTransDelete (in some earlier versions of AX it was just called “DatabaseTransDelete”).
    2. Make absolutely sure you’re currently in the company for which you want to delete the transactions!
    3. Run the class found in step 1. It will prompt you to confirm that you wish to remove the transactions. Again, make absolutely sure that the company it asks about is the one for which you want to delete the transactions!
    4. Let the task run. This can take quite a while if you have many transactions.
    5. Once it’s done, return to the Organization administration / Setup / Organization / Legal entities form. Make sure you’re not in the company you want to delete at this point, as you can’t delete the current company.
    6. Select the company you want to delete and press the “Delete” button (or Alt+F9).
    7. Confirm that you want to delete the company. This will also take a while, as it’s now deleting all the non-transactional data in the company.
    8. Sit back, relax and revel in the glory of a job well done! 🙂

Convert a real to string with all decimals in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

Every once in a while, I need to convert a real number to a string. Usually, just passing it to strFmt() is enough, but that function always rounds off to two decimals, which is not always what I want.

Then there’s the num2str() function, which does work well, but requires you know ahead of time how many decimals and characters you want.

What if you just want the number converted to a string, with all digits and decimals? For some reason, this is something that always has me Googling because it is surprisingly obscure to do and I do it so rarely that I usually can’t remember exactly how – in most languages, I would expect that you could just concatenate the real to an empty string, but X++ doesn’t support that.

Anyway, the by far easiest way I have found to do this is by using a .NET call. There are multiple options here as well, with and without options for advanced formatting, but if you just want the really simple conversion of a real to a string, this will suffice:

stringValue = System.Convert::ToString(realValue);

If this code is to be run on the AOS (for example in a batch job), remember to assert the necessary code access permission first. In this case you’ll need an InteropPermission of type ClrInterop to call .NET code, so the full code example would look something like this:

new InteropPermission(InteropKind::ClrInterop).assert();
stringValue = System.Convert::ToString(realValue);
CodeAccessPermission::revertAssert();

Be aware that this simple System::Convert function uses the system’s current locale with respect to decimal point character. This may not be an issue for you, but for me who lives in an area where comma is used rather than period as decimal separator, it may require further processing if the string for example needs to be used in a file that must be readable by other systems.

Replacing a failed drive in an mdadm array on Ubuntu

The information in this post is based Ubuntu 18.04 and the version of mdadm included in its repositories; at the time of writing v4.1-rc1. It may or may not be valid for other versions.

I recently had a sudden drive failure in my home file server, which consists of nine drives in an mdadm RAID-6 array. That’s always scary, but I was fortunately able to quickly source a replacement drive that was delivered already the next day so I could get the rebuild started.

I was admittedly a bit too cheap when I originally setup the file server; only two of the drives are actual NAS drives (Seagate IronWolf), while the rest are desktop drives (Seagate Barracuda). Not surprisingly, it was one of the desktop drives that had given up (after almost three years of service, though). It was completely dead; after moving it to a desktop USB enclosure all I got out of it was an unnerving clicking sound and neither Ubuntu 20.04 nor Windows 10 was able to detect it.

Oh well, on to the replacement part (and yes, the new drive I bought was an IronWolf, lesson learned) – as scary as it is losing a drive in a running array, it’s even scarier if you don’t know the correct procedure for replacing it. It’s not the first time I’ve had to replace a failed drive in an mdadm array, but fortunately it’s so rare that I usually have to look up the proper commands. This time I decided to whip up my own little guide for future reference.

So, first of all, when you get the dreaded fail event e-mail from mdadm, you need to identity which drive has failed. Sure, it will tell you the device name (in my case /dev/sdf), but it’s probably not obvious which physical drive that actually is as those names can change when the machine is booted.

If you’re not even sure which device name has failed, you can use the following command to find out (replace /dev/md0 with your RAID device):

mdadm -–query -–detail /dev/md0

As mentioned, in my case it was /dev/sdf, so let’s continue with that.

Then, you can try to find the serial number of the failed drive by issuing this command (if smartctl is not found, you need to install the smartmontools package on Ubuntu):

smartctl -–all /dev/sdf | grep -i 'Serial'

The serial number can then be compared to the serial numbers on the physical label on the drives to figure out which one has failed.

This time, I wasn’t so lucky, though. The drive was completely dead and even refused to provide SMART or other data, including the serial number.

Since I had physical access to the server (which you really need if you’re going to replace a physical drive yourself, I suppose ;-)) and the server was actually running when the disk failed (and continued to run fine thanks to the RAID-6 redundancy), I went with the really primitive (but actually highly effective and obvious) method of simply copying a large file to the server and watching which HDD light didn’t flicker. Within a few seconds I had identified the culprit.

Now, before yanking out the physical drive, it’s a good idea to formally inform mdadm of this intent, by issuing this command (replace device names with your own as appropriate):

mdadm -–manage /dev/md0 -–remove /dev/sdf1

On success, mdadm will reply will reply with a message saying that it “hot removed” the drive, apparently because the virtual raid device is actually running at the time.

If it fails with an error message similar to “device or resource busy”, it may be that mdadm has in fact not registered the drive to have completely failed. To make it do that, issue this command (again, remember to replace device names with your own as appropriate):

mdadm --manage /dev/md0 --fail /dev/sdf

After that, you should be able to remove the device from the array with the previous command.

Now it’s time to actually replace the disk. If you’re really, really – like, really – certain your machine and controller supports hotswap, you could do this without shutting down the machine. That would be the way to go on critical production systems running on real, proper server hardware that you know for a fact can handle it. My home file server is based on a consumer grade desktop mainboard with a couple of semi-noname SATA controllers in the PCIe slots to provide more SATA ports, though. Although SATA generally should support hotswap, I wasn’t about to risk anything in this setup, so I opted for shutting down the machine while replacing the disk.

Before doing that, it’s a good idea to comment out the raid device in the /etc/fstab file so that Ubuntu won’t try to mount it automatically on the next boot, because it might hang and force you into recovery mode due to the degraded RAID array. That may not be a big issue if it’s a desktop system, but I run this server headless without monitor or keyboard attached, so this would be a bit of a hassle.

After booting the machine with the shiny new drive installed, use lsblk or some other means to identify it. If you haven’t changed anything else, it will probably (but not necessarily) get the same name as the drive you replaced. In my case it did, so the new one is also called /dev/sdf.

As my array is based on partitions rather than physical devices, I needed to copy the partition table from a working drive to the new drive in order to make sure they’re exactly the same. If you run your array on physical devices instead, you can skip this step.

I used gdisk for this purpose, copying the partition table from /dev/sdc to /dev/sdf. Make sure to replace device names to match your own as appropriate. Notice the order here: you list the “to” drive first! This is a bit counter-intuitive for me, but just make sure you get it right so you don’t get another drive failure in the array 😉

sgdisk -R /dev/sdf /dev/sdc

Then to avoid UUID conflicts, generate new UUIDs for the new drive:

sgdisk -G /dev/sdf

And now finally the time has come to add the new drive to the array and get the rebuild party started! (Okay, it’s not really a party, it’s actually a quite slow and unnerving process as you really, really don’t want another drive failing at this time. Beer might help, though.)

Anyway, to add the new drive to the array, issue this command (again, make sure to replace device names with your own as appropriate):

mdadm -–manage /dev/md0 -–add /dev/sdf1

If all goes well, the drive will be added to the array without hiccups. I believe it is actually added as a “hot spare” by default, but since this array is missing a disk (the one that failed), it is immediately put into use and the rebuild process will start.

You can keep an eye on it like so:

watch cat /proc/mdstat

This will probably take a while; on my lowly server (based largely on consumer grade hardware and desktop drives, mind you) it was able to reach just under 100 MB/sec. Bear in mind that this is RAID-6, so there’s a lot of parity calculations involved with a rebuild; a RAID-10 would have been much faster. This particular machine has an AMD A10 9700E quad core CPU (the “E” meaning that it’s an under-clocked energy efficient model, i.e. not super fast), just to give you an idea of what to expect. With the nine 8 TB drives in my setup, the full rebuild took just over 24 hours.

During the rebuild, you can mount the filesystem on the array and use it like normal if you wish, but I prefer to leave it to the rebuilding until it’s done. Bear in mind that if one drive fails, another may soon follow, so you want the rebuild to be done as fast as possible as you really don’t want another drive to fail during that. Therefore, don’t burden it with other IO that isn’t strictly necessary.

Once it’s done, add it back to your /etc/fstab file, reboot and enjoy your files 😉

How to force kill a process in GNU/Linux

The information in this post is based on Ubuntu 20.04. It may or may not be valid for other versions.

Every now and then you have a hanging process that just won’t quit for some reason. The last time it happened for me was with the VLC media player, but it has happened with other programs too.

Unfortunately (or fortunately?) it doesn’t happen often enough for me to actually remember what to do about it each time, so I decided to write this little guide.

First, you need to find the process ID (PID) of the process. If the process is from a command-line program you can usually search for its executable name, but if it’s a desktop program it may not always be obvious what the name of the executable is, so you may need to do a bit of research.

In my case it was vlc which was obvious enough, though.

To get the PID you need to type:

ps aux | grep vlc

Which will show you any running process with “vlc” in the name.

Then you need to run the kill -9 command with root privileges on the PID you found:

sudo kill -9 PID

(replace “PID” with the number found with the first command)

And that’s it 🙂

Using a query in a SysOperation data contract class in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

I always seem to forget the details on how to specify and initialize a query in the SysOperation framework. I guess that most of the batch jobs I’ve been making aren’t based on user-configurable queries, but every now and then I do need to make such a batch job, so this post is also for my own reference, or memory-aid if you will 😉

First, in the data contract class, the query will be stored packed in a string. Its parm method must be decorated with the AifQueryTypeAttribute attribute, like so (in this example I’ve used the SalesUpdate query, but you can replace this with any AOT query):

[   
    DataMemberAttribute,
    AifQueryTypeAttribute('_packedQuery', queryStr(SalesUpdate))
]
public str parmPackedQuery(str _packedQuery = packedQuery)
{
    ;

    packedQuery = _packedQuery;
    return packedQuery;
}

If you want the query to be decided by the controller class instead, you can also use an empty string. In that case, you also need to implement a couple of helper methods (which you probably should implement anyway for your own convenience when you need to access the query):

public Query getQuery()
{
    ;

    return new Query(SysOperationHelper::base64Decode(packedQuery)); 
}
public void setQuery(Query _query)
{
    ;

    packedQuery = SysOperationHelper::base64Encode(_query.pack());
}

If you need to initialize the query (for example, add ranges), you should implement an initQuery method

public void initQuery()
{
    Query queryLocal = this.getQuery();
    ;

    // add ranges, etc...

    this.setQuery(queryLocal);
}

You need to make sure to call this method from the controller class.

Error “No metadata class defined for data contract object” in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

I recently encountered the somewhat cryptic error message “No metadata class defined for data contract object” when trying to start a SysOperation controller class.

After a bit of investigation, it turns out that the cause of this was that I forgot to decorate the ClassDeclaration of the data contract class with the [DataContractAttribute] attribute.

It seems there are a couple of other possible causes, but the above is the by far most likely one. Strange that I haven’t encountered it before, but I guess I’ve never forgotten that attribute before, then 😉

Hereby noted for future reference 🙂

String formatting with macro and strFmt in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

I recently encountered an issue with the strFmt function that baffled me for a bit. The most baffling part was that I by some weird coincidence have never encountered it before in my many years as an Axapta/Dynamics AX developer.

The issue was that I tried to use a macro as the format string for the strFmt function and it just didn’t work. It completely ignored the % parameters and only returned the remainder of the string.

After looking into it, I discovered that macros themselves can be used to format strings, which was also something I didn’t know. Oh well, it’s always good to learn something new, but I was still very surprised that I hadn’t happened to come across this before.

Basically, something like this:

#define.FormatMacro('%1-%2-%3')
; info(strFmt(#FormatMacro, salesId, itemId, lineNum));

will not work because the % signs in the macro are actually used for the macro’s own string formatting features. In this case, the strFmt function will see the formatting string as “–” and will therefore only return that.

Something like this:

#define.FormatMacro('%1-%2-%3')
; info(#FormatMacro(salesId,itemId,lineNum));

will work, but probably not the way you want it to. Instead of outputting the values of the three variables, it will output the names of the variables instead, in this case “salesId-itemId-lineNum”. (Notice that I didn’t put spaces after the commas when passing parameters to the macro, as I usually do in method calls. That’s because the macro will actually use such spaces as well, so the output would be “salesId- itemId- lineNum” if I did).

To actually use a macro as formatting string with strFmt, you need to escape the percentage signs with backslashes, like this:

#define.FormatMacro('\%1-\%2-\%3')
; info(strFmt(#FormatMacro, salesId, itemId, lineNum));

This will actually work as if you had supplied the format string directly.

This little job and screenshot of its output illustrates the examples:

static void StrFmtMacroTest(Args _args)
{
    #define.FormatMacro('%1-%2-%3')
    #define.FormatMacroEscaped('\%1-\%2-\%3')

    SalesId salesId = '1';
    ItemId  itemId  = '2';
    LineNum lineNum = 3.00;
    ;

    info(#FormatMacro(salesId,itemId,lineNum));
    info(strFmt(#FormatMacro, salesId, itemId, lineNum));
    info(strFmt(#FormatMacroEscaped, salesId, itemId, lineNum));
}
Infolog showing output of job to test strFmt with macros

Using the SysExtension framework to find out which subclass to instantiate in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

When implementing processing classes in Dynamics AX, you’re often faced with creating a class hierarchy in which each subclass corresponds to an enum value or has some other data coupling. A classic design is to then have a construct method in the superclass, which has a switch that determines which class to instantiate based on the input.

This works well in principle, but if you have many different possible inputs (many elements in an enum or perhaps the input is a combination of several different values), it can become tedious and error-prone to maintain and the design always has the disadvantage that you will need to modify said construct method if you ever add a new subclass or make changes to which subclass should be used based on which input.

Fortunately, there is a much more elegant, but unfortunately also much less known, way of doing this, namely by use of the SysExtension framework.

This framework takes advantage of attributes that you can use to decorate your subclasses to make the system able to figure out which subclass should be used for handling what. You will still need a construct method, but if done correctly, you will never have to modify it when adding new subclasses.

Let’s look at an imaginary example and say that you’re going to implement a hierarchy that does some sort of processing based on the InventTrans table. Which processing to do depends on the StatusReceipt and StatusIssue of the records, as well on whether the records are related to SalesLine, PurchLine or neither. Already now, you’re looking at a lot of different combinations. Let’s then say that you know that for now you only need to handle a handful of the combinations, but you also know that you will be asked to be able to handle more and more combinations over time.

Let’s keep it relatively simple and say that for now you only need to handle records related to SalesLine with a StatusIssue of ReservPhysical or ReservOrdered, all other combinations can be ignored for now, but since you know you will have to handle them later, you’ll want to design your code in a way that makes it easily extensible.

Your hierarchy may look something like this for now:

    • MyProcessor
      • MyProcessor_Sales
        • MyProcessor_Sales_ReservOrdered
        • MyProcessor_Sales_ReservPhysical

Now, you could easily implement a method in the superclass that instantiates a subclass based on a ModuleInventPurchSales and a StatusIssue enum. But you will then need to modify the superclass every time you add a subclass, and that is not really the idea of inheritance in object-oriented programming. Afterall, you don’t need to modify RunBaseBatch or SysOperationServiceBase every time you add a new batch job.

Instead, you can make use of the SysExtension framework. That will require you to add another class, which needs to extend SysAttribute. This class will be used as the attribute you can decorate your processing classes with.

This class is very similar to how you would make a data contract class for a SysOperation implementation, in that it will have some data members and parm methods for getting and setting those values.

In our case, the ClassDeclaration may look something like this:

class MyProcessorSystemAttribute extends SysAttribute{
    ModuleInventPurchSales  module;
    StatusIssue             statusIssue;
    StatusReceipt           statusReceipt
}

You need to make a new() method for instantiating all data members. If you wish you can give some or all of them default values, but I haven’t done that.

public void new(ModuleInventPurchSales  _module,
                StatusIssue             _statusIssue, 
                StatusReceipt           _statusReceipt)
{
    ;

    super();

    module          = _module;
    statusIssue     = _statusIssue;
    statusReceipt   = _statusReceipt;
}

And you should also implement a parm method for each data member, but I’ve omitted those here as I’m sure you know how to do that – otherwise, let’s consider it an exercise 😉

Now you can use your attribute class to decorate each of your processing classes. For example, the class declarations could look like this:

[MyProcessorSystemAttribute(ModuleInventPurchSales::Sales,
                            StatusIssue::None,
                            StatusReceipt::None)]
class MyProcessor_Sales extends MyProcessor
{
}
[MyProcessorSystemAttribute(ModuleInventPurchSales::Sales,
                            StatusIssue::ReservOrdered,
                            StatusReceipt::None)]
class MyProcessor_Sales_ReservOrdered extends MyProcessor_Sales
{
}
[MyProcessorSystemAttribute(ModuleInventPurchSales::Sales,
                            StatusIssue::ReservPhysical,
                            StatusReceipt::None)]
class MyProcessor_Sales_ReservPhysical extends MyProcessor_Sales
{
}

You can of course name your classes any way you want, the important part here is that you decorate your classes with attributes that correspond to what kind of processing they do. (But bear in mind that there are naming conventions for class hierarchies in Dynamics AX and it’s always a good idea to follow those, if possible).

Now that you have decorated your classes to identify what kind of processing each of them does, you can take advantage of the SysExtension framework to instantiate objects of the subclasses as needed.

In your superclass (MyProcessor), you could add a construct method like this:

public static MyProcessor construct(ModuleInventPurchSales _module,
                                    StatusIssue _statusIssue,
                                    StatusReceipt _statusReceipt)
{
    MyProcessor                 ret;
    MyProcessorSystemAttribute  attribute;
    ;

    attribute = new MyProcessorSystemAttribute( _module,
                                                _statusIssue,
                                                _statusReceipt);

    ret = SysExtensionAppClassFactory::getClassFromSysAttribute(
                                        classStr(MyProcessor),
                                        attribute);

    if (!ret)
    {
        //  no class found
        //  here you could throw an error, instantiate a default
        //  processor instead, or just do nothing, up to you
    }

    return ret;
}

The really interesting part – and really the object (pardon the pun) of this entire post – is the getClassFromSysAttribute() method in the SysExtensionAppClassFactory class. What this method does is that it accepts the name of the superclass of a hierarchy (and this superclass does not need to be at the top of the hierarchy; it simply means that only classes extending this class will be eligible) and an attribute object. It then returns an object of a class that extends the specified superclass and is decorated with a corresponding attribute.

You can obviously add as much further validation or logic to the construct method as you wish to, but the important takeaway here is that once implemented, you should never have to modify this method again. You can add subclasses to the hierarchy and as long as you make sure to decorate them appropriately, the construct method will find them even though they didn’t exist when it was written.

What about performance? I honestly haven’t attempted to benchmark it, but my gut feeling is that this probably performs worse than the classic switch statement design. However, considering that by far the most performance issues in Dynamics AX are caused by database access (either due to poor database design or poor code design when accessing the database), I wouldn’t worry too much about it.

Of course, if you’re implementing something that will require thousands of objects to be created quickly, you may want to investigate further, but in the classic cases where you just instantiate a single object to do some lengthy processing, I doubt it’ll matter. Also, considering my troubleshooting tip (next paragraph), it appears that the SysExtension framework relies on caching, so in a running system I doubt it has a significant performance hit.

Troubleshooting: If the construct method doesn’t find your subclasses even if you’re certain they’re decorated correctly, it may be a caching problem. Try clearing caches on both client and server. It shouldn’t be necessary to actually restart the AOS, but it may be last resort.

Evnovo/Artillery Sidewinder X1 setup in Ultimaker Cura

The information in this post is based on Ultimaker Cura 4.7.1. It may or may not be valid for other versions.

As of Cura 4.7.1, the popular Artillery Sidewinder X1 printer is still not supported out of the box. This is my first (and so far only) 3D printer and I’m quite happy with it, but it’s always a bit annoying to have to find and install custom profiles when Cura is updated, so I’ve been stuck on a much older version for a long time.

However, I recently discovered that the Sidewinder X1 works perfectly well with Cura’s “Custom FFF printer” profile as long as you make a few adjustments, like so:

When you add a printer, choose “Add a non-networked printer” and then under “Custom” choose “Custom FFF printer”.

Cura's "Add a printer" dialog.
Cura’s “Add a printer” dialog.

Then click “Add”, which takes you to the Machine Settings dialog.

Here you need to change the size (X = 300, Y = 300 and Z = 400) of the printer, tick the checkbox for Heated bed and make sure to set the G-code flavor to “RepRap”. The rest should be good at defaults, but check to make sure it’s set like in this screenshot

Cura's "Machine Settings" dialog.
Cura’s “Machine Settings” dialog.

Then change to the “Extruder 1” tab page, where you need to make sure that “Compatible material diameter” is set to 1.75 mm (it defaults to 2.85 mm). If you use a non-standard nozzle size you will also need to change that here, but in my case the printer had a 0.4 mm nozzle at this time, so I didn’t change it:

Cura's "Machine Settings" dialog on the "Extruder 1" tabpage.
Cura’s “Machine Settings” dialog on the “Extruder 1” tabpage.

Click “Next” and you’re done 🙂

How to iterate over the elements of an enum from X++ code in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

I was recently creating a form that needed to display a value for each element in an enum. Rather than manually creating the fields (and then needing to maintain the form if the enum is ever modified), I decided to implement it dynamically so that it would automatically add the fields to the design at run time.

However, I soon discovered that actually iterating over the values in an enum, while easy enough once you know how, is a bit confusing.

You obviously need to start with the DictEnum class. As you will see, this class has several methods for obtaining information such as name and label from both index and value.

The difference between index and value is that index is an element’s number in the enum, if the enum’s elements were numbered sequentially starting from zero, while value is the element’s actual “value” property. As most enums have values numbered sequentially from 0, the index and value of an element will often be the same, but certainly not always.

But how do you know which values an enum has? This is where it gets confusing. The DictEnum class has a method called values(). You might expect this method to return a list of the enum’s values, but that would obviously be too easy, so instead it returns the number of values the enum contains. However, the number of values has nothing do with the actual values, so you need to use this number as a basis for calling the index-based methods, not the value-based once. If only they had named this method indexes() instead, it would have been less confusing 😉

Also bear in mind that enum values (and apparently these “indexes”) start with 0, unlike array and container indexes in X++, which start with 1, so to loop over the elements in an enum you could do something like this:

DictEnum dictEnum = new DictEnum(enumNum(SalesStatus));
Counter  c;
;

for (c = 0; c < dictEnum.values(); c++)
{
    info(strFmt('%1: %2', dictEnum.index2Symbol(c),
                          dictEnum.index2Label(c)));
}

This will output the symbol and label of each element in the enum to the infolog.

The difference between data() and buf2Buf() in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

When you need to copy the value of all fields from one table buffer to another in Dynamics AX, you would traditionally do something like:

toTable.data(fromTable);

This works well and in most cases is the way to go.

However, you also have the option of using the buf2Buf function instead:

buf2Buf(fromTable, toTable);

This works well too. So what’s the difference?

The difference is that buf2Buf does not copy system fields. System fields include fields such as RecId, TableId, and perhaps most importantly in this context, DataAreaId. The reason the latter is the most important is that the most typical case where you would use buf2Buf() instead of data() is when duplicating records between company accounts, typically by use of the changeCompany keyword.

For example, if you’re in the “dat” company and has another company called “com” that you wish to copy all records in CustTable from:

while select crossCompany : ['com'] custTableFrom
{   
    buf2Buf(custTableFrom, custTableTo);
    custTableTo.insert();
}

In this case, it will work because buf2Buf copies all field values, except system fields to the new buffer. Had you used data() instead, the new record would have been inserted in the “com” company accounts because that value would have been copied to the new buffer as well. (Actually, it would have resulted in a duplicate key error, but that’s not what you want either).

Shrink virtual disk in VMware Player

The information in this post is based on VMware Player 7 running on a 64 bit Windows 7 host with a 32 bit Windows 7 guest. It may or may not be valid for other versions and platforms.

I usually don’t bother much with managing the size of my virtual machine disks as I store them in a location with plenty of space, but I recently had a Windows 7 virtual machine go on a rampage with its disk and grow the virtual disk file to almost 125 GB, even though it reported the actual used size of the disk to only be about 20 GB. I think the problem was caused by some issues with Windows Update that would sometimes churn on the hard disk for several hours, but I’m not sure of that.

Anyway, a virtual disk this size was getting to be a bit of a pain to backup daily, so I decided to attempt to reclaim the free space and shrink the disk file. To my surprise, the shrink option in VMware Player finished very quickly, stating that the disk had been successfully shrunk – but on inspection, it actually still took up exactly the same amount of space.

After some research I discovered that you need to run a VMware tool from within the virtual machine to mark all the free space as something that can be reclaimed. This step was complicated by the fact that most of the documentation I found mentioned an option in the VMware Tools GUI inside the host, but that option does not exist in current versions of VMware Tools. Instead, you need to rely on a command prompt utility.

This will temporarily grow the size of the virtual disk to the maximum allowed size, so make sure to store it on a hard disk with sufficient space for this. Also make sure to have enough time to complete it, as you should not interrupt it during shrinking or you risk permanent damage to your virtual machine (making a backup before doing this is highly recommended regardless).

The exact procedure turned out to be:

1. Defragment the disk from within the guest operating system.

2. Still inside the guest, start a command prompt with elevated privileges (Run as Administrator on Windows) and execute the following commands (obviously replacing “c:\” with the drive you actually want to shrink):

cd "C:\Program Files\VMware\VMware Tools"
vmwaretoolboxcmd disk shrink c:\

This will run for quite a while, depending on the size of your virtual disk and the speed of the physical hard drive it resides on. A rough estimate of needed time is probably something along the lines of the time it would take to write the maximum size of the virtual disk twice – so, for example, if your virtual machine is configured with a maximum hard disk of 128 GB, expect the running time of this to be roughly the time it would take to write 256 GB to the hard disk in question.

3. Shut down your virtual machine and verify that the vmdk file is now closer to what you’d expect. If this is not the case, you may need to use the defragment and shrink options from the VMware Player GUI as well, but when I did it it automatically shrunk the disk file after the above command finished.

As a final note: if your guest operating system is Linux (say, Ubuntu) instead of Windows, the procedure is pretty much the same. The command is just called vmware-toolbox-cmd instead and is located in the /usr/sbin directory by default. So, to shrink the root file system you might call something like this:

cd /usr/sbin
vmware-toolbox-cmd disk shrink /

Other than that, it works the same 🙂

Combine multiple virtual disk files into single vmdk with VMware Player

The information in this post is based on VMware Player 7 running on a Windows 7 64 bit host. It may or may not be valid for other versions and platforms.

I recently received a virtual machine that I did not get to install myself. I’m as picky as most older nerds *cough* about how my stuff is set up, and one thing that particularly annoyed me about this VM was that the person who had installed it had opted to have the virtual disk stored in multiple smaller files instead of one large. On modern file systems with no practical upper file size limit I really can’t see the advantage of this.

Anyway, I decided to see if I could get the virtual disk files (with vmdk extension) merged into one. Unfortunately, VMware Player does not come with an option to do this, you need one of the “bigger” VMware products, such as Workstation.

Fortunately, I discovered that a small command line utility exists and you can download it for free from VMware’s website (search for KB article 1023856 in their Support section). Unzip the utility to the folder where you have VMware Player installed.

Now, let’s assume you have a virtual machine where the disks are stored as virtualdisk.vmdk, virtualdisk-s001.vmdk, virtualdisk-s002.vmdk, virtualdisk-s003.vmdk, etc. which you wish to merge into a single large file instead. To do this, make sure the virtual machine is shut down, then open a command prompt on the host and execute the following command (obviously replacing paths and filenames for those valid on your system):

"C:\Program Files (x86)\VMware\VMware Player\vmware-vdiskmanager.exe"
-r "D:\VM\virtualdisk.vmdk" -t 0 virtualdisk-combined.vmdk

After the utility finishes, move all the old vmdk files to a separate folder (as backup until you’re sure the new file works), then rename the virtualdisk-combined.vmdk file to virtualdisk.vmdk (again, use the actual names of files on your system instead) and attempt to start up the virtual machine in VMware Player. For me it worked without a hitch 🙂

Once you have verified that it works, you can delete the old vmdk files to save some disk space.

Temporarily toggle admin mode in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

Sometimes – especially when testing security elements – it is convenient to be able to temporarily disable your own administrator privileges. Of course, you could just remove yourself from the Administrators group, but that holds a very real risk of you locking yourself out of the system and needing assistance to get back in.

If you just need to be without administrative privileges temporarily in a single session, you can execute the below job instead. It simply toggles back and forth between admin mode, so just run it again to regain your administrator awesomeness.

static void ToggleAdminMode(Args _args)
{
    ;
    
    SecurityUtil::sysAdminMode(!SecurityUtil::sysAdminMode());
    info(strFmt('Admin mode: %1', SecurityUtil::sysAdminMode()));
}

Toggling administrative privileges this way also has the advantage that if you mess up and lose access to something you need, you can just restart the Dynamics AX client, at which point your privileges will be restored to normal.

Hide SysOperation data contract dialog field in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

If you need to hide a data contract field from an automatically generated SysOperation dialog, you simply need to apply the SysOperationControlVisibilityAttribute attribute to the corresponding parm method. For example:

[DataMemberAttribute,
SysOperationControlVisibilityAttribute(false)]
public ItemId parmItemId(ItemId _itemId = itemId)
{
    ;

    itemId = _itemId;
    return itemId;
}

Create account string with ledger dimensions from code in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

The way ledger dimensions work changed drastically with the release of Dynamics AX 2012. You have probably seen the dynamic account strings, in which main account numbers are merged with ledger dimensions into one string. This is handled automatically by the user interface, but as a programmer you may have to do the same from code.

The underlying data structures are rather complex, but fortunately the utility class AxdDimensionUtil can help you with this. Assuming you have a main account ‘0001’, a dimension called “Place” with a value of “Home”, and a dimension called “Purpose” with a value of “Expenses”, you can create a container like this:

dimensions = ['0001', '0001', 2, 'Place', 'Home', 'Purpose', 'Expenses'];

Notice how the account number is repeated; the first element is actually the display value, I just use the account number for both (I’m actually not entirely sure where the display value is used).

The third element in the container is the number of dimensions, two in this case. Then the dimensions with ID followed by value.

Finally, you can pass the container to the AxdDimensionUtil::getLedgerAccountId method, which will return a ledger dimension record ID you can use as reference on the records where you need it, for example on ledger journal transactions:

ledgerJournalTrans.LedgerDimension = AxdDimensionUtil::getLedgerAccountId(dimensions);

The actual display of the merged account string is still handled by the user interface, but this method can help you create the underlying data without getting too gritty with the details.

Update cross reference in batch in Dynamics AX 2012

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

An updated cross reference is an invaluable tool for some tasks such as debugging and searching for where specific elements are used in Dynamics AX. Unfortunately, it takes so long to update that it is done very infrequently on many projects – often it is up to the development lead to do it and let his or her session hang overnight. If it crashes or gets interrupted in the meantime it’s all over again.

Fortunately, running it as a batch job is very simple. Assuming you want to clear (truncate) any existing data in the cross reference tables and update the cross reference for all elements (full tree), just run this little job:

static void CrossRefUpdateBatch(Args _args)
{
    ;

    xRefUpdate::truncateXrefTables();
    xRefUpdateIL::updateAllXref(true, false, true);
    info("Done, cross reference update batch job created.");
}

It should be pretty self-explanatory. The three boolean parameters for the updateAllXref method just means to update full tree, not update selectively (as we want full tree), and to update table relations as well. You should see the job complete very quickly, already giving you a hint that the actual task of updating has been passed to the batch processor so you don’t have to wait for it 🙂

Open up the batch jobs form and verify that a batch job with the description “Update xref for complete AOT” has been created. In fact, it’s probably already running. If you want it to repeat automatically, say, once a week, you can just edit the batch job to fit your needs. Or, you can just run the above code snippet whenever you need to update the cross reference again.

Notice that the batch job is created to run in the empty batch group by default. If you need it to be executed in a different group or on a specific AOS you may need to do some tweaking.

You can close down the AX client and go home, just make sure the batch server AOS is not restarted until the batch job has finished 😉 – depending on your hardware, a full tree cross reference update typically takes between 8 and 12 hours.

Dynamics AX 2012 SysOperation framework quick overview

The information in this post is based on Dynamics AX 2012 R3. It may or may not be valid for other versions.

This post is just meant as a quick overview and cheat sheet. If you are new to the SysOperation framework, I strongly suggest that you read Microsoft’s white paper on the subject as well. The information here may be useful if you just need a quick brush up on the different classes involved in developing operations with this framework.

There are variations, but when I use the framework I typically implement three classes:

      • Data contract (should extend SysOperationDataContractBase)
      • Service (should extend SysOperationServiceBase)
      • Controller (must extend SysOperationServiceController)

Additionally, I may also implement a UIBuilder class (must extend SysOperationUIBuilder), but that is only necessary if the dialog for some reason has to be more advanced than what the framework generates automatically.

Data contract

The data contract holds the data members needed for your operation. It can be compared to the typical CurrentList macro defined in the RunBase framework, but implemented as a class instead. The data contract should extend SysOperationDataContractBase, but will work even if it doesn’t. The advantage of extending the super class is that it provides some session information that may be handy.

[DataContractAttribute]
class MyDataContract extends SysOperationDataContractBase
{
    ItemId itemId;
}

In this example, the itemId is a data member. You need to implement a parm method for each data member and tag it with the DataMemberAttribute so the framework knows what it is. This enables the framework to automatically build the dialog for you.

[DataMemberAttribute]
public ItemId parmItemId(ItemId _itemId = itemId)
{
    ;

    itemId = _itemId;
    return itemId;
}

Service

The service class is the class that contains the actual business logic. It is not concerned with showing dialogs, batch processing or anything of the sort – that is the responsibility of the controller class. By separating this, you are more likely to design your code well and make more reusable code.

Like the data contract class, the service class does not need to inherit from anything in particular, but it should inherit from the SysOperationServiceBase class, at least if you expect that the service will be run as a batch job, as the super class provides some information about the batch context . The method that starts the operation (i.e. runs the business logic) must take an object of your data contract class as input and should be decorated with the [SysEntryPointAttribute]. For example:

class MyService extends SysOperationServiceBase
{
}

with a method called run:

[SysEntryPointAttribute]
public void run(MyDataContract _dataContract) {     // run business logic here }

Controller

The controller class handles the execution and batch processing of your operation. It also makes sure that the code is executed in CIL for maximum performance. The controller class typically inherits from the SysOperationServiceController class, although there are other options as well.

class MyController extends SysOperationServiceController
{
}

The constructor of the super class takes a class name, method name and (optionally) execution mode as parameters. The class and method names should be the name of your service class and the method that should be run on it. So, you might implement your controller’s construct method like this:

public static MyController construct()
{
    ;

    return new MyController(classStr(MyService),
methodStr(MyService, run)); }

Then the main method of the MyController class can be as simple as

public static void main(Args _args)
{
    ;

    MyController::construct().startOperation();
}

And you’re basically done. The above is obviously a very simple example and the framework contains a plethora of other options and possibilities, but this serves as a quick overview if you need a brush up when you haven’t used the framework for a while.

How to set up a firewall on Ubuntu Server

The information in this post is based on Ubuntu Server 14.04 x64. It may or may not be valid for other versions.

When I first started out with Linux (Ubuntu) servers, setting up a firewall involved manually creating and maintaining a potentially complex configuration file for iptables. However, I have recently discovered ufw, which is short for Uncomplicated Firewall – and it really is 🙂

My installation of Ubuntu Server 14.04 already had ufw installed, but if your doesn’t, simply install it from the repositories:

sudo apt-get install ufw

UFW is actually just a tool that simplifies the iptables configuration – behind the scenes, it is still iptables and the Linux kernel firewall that does the filtering, so ufw is neither less nor more secure than these. However, because ufw makes it a lot easier to configure a firewall correctly, it may reduce the risk of human error and is therefore possibly more secure for inexperienced admins.

If your server is configured with IPv6 as well as IPv4, make sure that this is enabled for UFW as well. Edit the file /etc/default/ufw and look for a line saying IPV6=yes. On my installation it was already there, but if it’s not or if it says no, you should edit it.

Then simply use the command prompt to enable the ports you want opened. If you are connected to your server via ssh, make sure to allow that as well or it may disrupt your connection and possibly lock you out of your server when you activate it – depending on whether you have physical access to the server or not, this may be kinda inconvenient 😉

For example, if you use ssh on the standard port 22 and you are configuring a web server that supports both unencrypted (HTTP on port 80) and encrypted (HTTPS on port 443) connections, you would issue the following commands to configure ufw:

sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

If you need more rules, simply add them as above.

If you have a static IP address and only need to be able to connect via ssh from the one location, you can also restrict ssh connections to a single origin address like this:

sudo ufw allow from 192.168.0.1 to any port 22

Of course, enter your own IP address instead.

When done, enable ufw by entering:

sudo ufw enable

And you’re done! The firewall is running and will automatically start up when you reboot your server 🙂

If you make changes to the ufw configuration, you may need to disable and enable it again to put them into effect, like this:

sudo ufw disable
sudo ufw enable

To look at the current configuration, simply enter:

sudo ufw status

If ufw is not enabled, this will simply show an “inactive” message, otherwise it will list the currently defined rules.

Match location based on file extension with NGINX

The information in this post is based on NGINX 1.4.6 running on Ubuntu Server 14.04 x64. It may or may not be valid for other versions.

I’m not all that good at regular expressions (something I should probably work on, I know), so I often need to read up on it when I have to do more than the very  simplest of pattern matching in for example NGINX’s location context.

One that is very useful if you need to handle specific file types differently is the ability to match a location based on the extension of the requested file. And it’s very easy too, your location directive could simply look like this:

location ~* \.(js|css|html|txt)$
{
    // do something here
}

Of course, you can just change the extensions to whatever you need.

The above example is case-insensitive (for example, it will match both .js and .JS). If you want it to be case-sensitive, just remove the * after the ~.

What you do with the match is up to you; typically, you’d rewrite it to a back-end that does some sort of preprocessing, or you may just want to read the files from other folders than what it looks like to the public, the possibilities are endless.

Deleting NGINX cache puts critical unlink errors in error log

The information in this post is based on FastCGI caching on NGINX 1.4.6 running on Ubuntu Server 14.04 x64. It may or may not be valid for other versions.

After migrating several sites from Apache to NGINX I have grown very fond of its built-in caching capabilities, which works extremely well under most circumstances without much meddling from me.

However, one thing I really can’t do without is the ability to clear the cache myself. The free community edition of NGINX only supports time-based cache expiry (i.e. you can set it up to check if something has changed after an hour, a day, etc.). But what if there is no reliable way of determining ahead of time when a certain resource will change? For example, I have no idea if it will be an hour, a day or a year before I come back and edit something in this post – and why only cache for an hour if caching for a day would have been fine?

This is where the ability to clear the cache manually (or by having your web application notify NGINX that something should be purged) is needed. The people behind NGINX are clearly aware of the need for this as the feature is supported in the paid version of their product – but while they are certainly entitled to set up their licensing any way they want, the price is a bit steep for me when this function is the only paid feature I really need.

Fortunately, it turns out you can just delete files from the cache directory yourself and NGINX will pick up on this and fetch a new copy from your back-end without a hitch. However, if you do this without tweaking your configuration you are likely to see a whole bunch of messages similar to this one in your error log after a while:

2015/03/04 17:35:24 [crit] 16665#0: unlink() "/path/to/nginx/cache/9/a0/53eb903773998c16dcc570e6daebda09" failed (2: No such file or directory)

It appears that these errors occur when NGINX itself tries to delete cache entries after the time specified by the inactive parameter of the fastcgi_cache_path directive. The default for this is only 10 minutes, but you can set it to whatever value you want. I’ve set it to 7 days myself, which seems to work well as I haven’t seen this error at all after changing it.

I find it really strange that it is considered a critical error that a cache entry cannot be deleted because it doesn’t exist. The fact that its severity classification is so high means that it’s impossible to get rid of just by ignoring log entries below a certain threshold. As soon as a new copy is fetched from the back-end the entry will exist again, so this should be a warning at most, in my opinion.

Now, if the cache entry couldn’t be deleted because of problems with permissions or something third, that would be a critical error, because it might make NGINX continue serving cached content long after its expiry time, but the clean-up process doesn’t seem to make this distinction.

FastCGI PHP_VALUE settings leaking to other sites on NGINX with PHP-FPM

The information in this post is based on NGINX 1.4.6 and PHP-FPM 5.5.9 running on Ubuntu Server 14.04 x64. It may or may not be valid for other versions.

Background

I was recently setting up a new website on a server that had a couple of other sites on it already. The server and all of the sites on it are owned and controlled by me, so even though it has multiple virtual hosts, I don’t really consider it a “shared hosting environment” as such.

More or less by chance, I suddenly noticed that the new site had some PHP settings that were certainly not default. On one of the other sites, I have a virtual host defined as an admin area where I’m allowed to bulk upload large files, so the PHP settings regarding file uploads and max POST size had been increased for the relevant location in that virtual host using the fastcgi_param PHP_VALUE directive – or so I thought, because it was now apparent that these settings were leaking and polluting other sites. I assume the same leak occurs if using fastcgi_param PHP_ADMIN_VALUE instead, but I haven’t actually tested it.

Specifically, the following PHP settings were increased for this location: post_max_size, upload_max_filesize, max_file_uploads, and max_input_time. Also, NGINX’s own client_max_body_size was increased to the same as PHP’s post_max_size.

While having way too high max POST size and upload size for publicly available areas is certainly not optimal, it’s not a huge security issue either (attackers could potentially attempt to exploit it to deplete resources on the server by posting massive requests) – but other settings I might have set could have been, so I’m glad I found out about this on something fairly innocent.

The Problem

It seems that the issue stems from the passed options getting applied to the PHP-FPM child process that happens to process the request and then remain there while that process lives, which can be for quite some time as one of the major performance improvements in PHP-FPM over traditional PHP execution is that each process handles multiple requests, so the entire engine doesn’t have to be started for every request.

After doing some research it is still unclear to me whether this is a bug, normal behaviour, or just an error in my setup. I have seen people claiming all three. The problem was completely reproducible to me, though – restarting PHP-FPM would return the settings to default, but as soon as I’d visited the virtual host with the special settings, those settings would start appearing on other sites on the same server. Regardless, the important part is how to fix it.

The Solution

After some further research and experimenting, it appeared to me that the only reliable way of fixing it was to divide the virtual hosts into separate PHP-FPM pools so that they would not share child processes. Many people will tell you that you should be doing that anyway for security reasons; while I can see both pros and cons here, I’m inclined to think they’re right based on my experiences with this problem.

I have decided to split the description of how configure separate PHP-FPM pools into a separate post, which you can read here.