On CachePolicy, UriBuilder and mobile .NET

In the Windows Phone 7 application I was working on there was a problem with some WebRequest caching. I'm not sure if it should have been fixed on the client side or server side, but since it's just a matter of setting proper CachePolicy property of WebRequest I was going to add it to mobile application.

Imagine my surprise when I found out that CachePolicy is not supported on Windows Phone 7. The obvious workaround is adding an URI param "no-cache=MS_FROM_EPOCH" or something like that.

Now, if you'd ask me what is the cardinal example of stuff that shouldn't be done using string concatenation, but is notoriously done that way, I'd answer: URI building. It's highly structured string with RFC, so it stands to reason to create and use a dedicated builder and parsers API to make sure everything is well-formed.

Let's take a look at UriBuilder class and URIs in general. All the stuff between ? and # (VERIFY) is called query string, which contains key-value pairs separated by &s. What kind of API for query string would make sense? An IDictionary<String, String>! What API is presented by UriBuilder? A string! Take a look at the documentation. Here's the best part:

Note Do not append a string directly to this property. If the length of Query is greater than 1, retrieve the property value as a string, remove the leading question mark, append the new query string, and set the property with the combined string.

Ridiculous. Seriously, MS, I don't care it's transformed to a string, it's a key-value dictionary, so let me treat it as such!

Fortunately C# has this nice feature called extension methods, meaning we can "add" a method to a class. There are no extension properties, but we can have a method like this:

UriBuilder WithQueryParam(this UriBuilder uri, String key, String value)

But now we need to check for duplicate keys and for that we have to parse existing query string. Fortunately there is a method for this: HttpUtility.ParseQueryString.

Except it's not available on mobile .NET. Again. But why?

The same thing irritate me when I worked on Blackberry apps and I had to put up with Java ME no String.format, no date handling methods, no generics and sane collections. But I understand that Java ME is supposed to work on wider range of devices, some of them with very limited CPU and RAM resources, so every cycle and every byte of memory matters.

Now let's take a look at Windows Phone 7 hardware requirements for manufacturers: 256MB of RAM and 800MHz CPU. I'm not an .NET expert, but I'd hazard a guess that supporting  full desktop .NET would not make any difference on those powerhouses. Hell, you should be able to send a man to the Moon using this hardware, so please, can I have a query string parser?

Is it some kind of Windows Mobile 6 legacy, or just a very bad MS joke that non-MS people don't get?

The quest for paramerized converter for Windows Phone 7

Warning: this is going to be a long post, because I want to provide full background for the issues I found. Also, none of the code posted here works in 100% of cases.

Recently I was involved in refactoring a program written by interns, which included an over engineered and messed up solution for application settings. I had plenty of time and despite of the messed up implementation the original idea might be useful for any form filled by user, so instead of scraping the whole thing I decided to clean it up.

Every type of setting is represented by different class derived from common base. The model of settings consist of a collection of base class objects. That collection is bound to ItemsSource property of ListBox and instead of single data template I use template selector, which creates different UI controls for different setting type.

There is a BooleanSetting, which is represented by a single ToggleSwitch from Silverlight from Windows Phone Toolkit. There is a ClosedListSetting, represented by ListPicker, which is used for a setting where user can choose one of many options from a predefined list (hence the name). Finally there is a OpenListSetting, which is just like a ClosedListSetting, except there is the "Other" option which allows user to enter any value. On web forms such field is usually represented as a radio button group with an input box beside the "Other" option.

For UI consistency I wanted to represent the OpenListSetting using ListPicker, and when the user would select the "Other" option, additional entry field will appear. It seemed really easy, I thought I'll just bind the visibility of the custom entry field to ListPicker's SelectedIndex property with a value converter.

The first complication is that the position of custom option might be different for a different settings - usually it's the last one, so it depends on the number of predefined options; for maximum flexibility it should be possible to have many custom options in one OpenListSetting. The obvious solution is to bind some value to converter's parameter property.

Quick search on Google reveals the major flaw in that plan, to wit, you cannot use the binding for converter parameter. There is a Multibinding, which sounds like a likely solution (after all I want to bind visibility to two properties and an operation between those properties), but apparently it's not available in Silverlight for Windows Phone 7. Yay, it's Java ME crap all over again!

The suggested workaround is creating a converter as a non-visual FrameworkElement, with parameter as a dependency property, which can be used in binding. I created a base class:

public abstract class ConverterWithBindingParameter<T, TConverter> : FrameworkElement, IValueConverter
{
    public static readonly DependencyProperty ParameterProperty = DependencyProperty.Register(
        "BindingParameter",
        typeof(T),
        typeof(TConverter),
        null);

    public T BindingParameter
    {
        get { return (T)GetValue(ParameterProperty); }
        set { SetValue(ParameterProperty, value); }
    }

    abstract public object Convert(object value, Type targetType, object parameter, CultureInfo culture);
    abstract public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture);
}

And then derived the concrete converter for a problem at hand:

public class EqualToVisibilityConverter : ConverterWithBindingParameter<int, EqualToVisibilityConverter>
{
    override public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
    {
        return (value.Equals(BindingParameter)) ? Visibility.Visible : Visibility.Collapsed;
    }

    override public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
    {
        throw new NotImplementedException();
    }
}

Edit XAML, rebuild, run, kaboom:

System.ArgumentException was unhandled
    StackTrace:
    at System.Reflection.RuntimeMethodInfo.InternalInvoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, StackCrawlMark& stackMark)
    at System.Reflection.RuntimePropertyInfo.InternalSetValue(PropertyInfo thisProperty, Object obj, Object value, Object[] index, StackCrawlMark& stackMark)
    at System.Reflection.RuntimePropertyInfo.SetValue(Object obj, Object value, Object[] index)
    at MS.Internal.XamlMemberInfo.SetValue(Object target, Object value)
    at MS.Internal.XamlManagedRuntimeRPInvokes.SetValue(XamlTypeToken inType, XamlQualifiedObject& inObj, XamlPropertyToken inProperty, XamlQualifiedObject& inValue)
    at MS.Internal.XcpImports.MeasureOverrideNative(IntPtr element, Single inWidth, Single inHeight, Single& outWidth, Single& outHeight)
    at MS.Internal.XcpImports.FrameworkElement_MeasureOverride(FrameworkElement element, Size availableSize)
    at System.Windows.FrameworkElement.MeasureOverride(Size availableSize)
    at System.Windows.FrameworkElement.MeasureOverride(IntPtr nativeTarget, Double inWidth, Double inHeight, Double& outWidth, Double& outHeight)
    at MS.Internal.XcpImports.Measure_WithDesiredSizeNative(IntPtr element, Single inWidth, Single inHeight, Single& outWidth, Single& outHeight)
    at MS.Internal.XcpImports.UIElement_Measure_WithDesiredSize(UIElement element, Size availableSize)
    at System.Windows.UIElement.Measure_WithDesiredSize(Size availableSize)
    at System.Windows.Controls.VirtualizingStackPanel.MeasureChild(UIElement child, Size layoutSlotSize)
    at System.Windows.Controls.VirtualizingStackPanel.MeasureOverride(Size constraint)
    at System.Windows.FrameworkElement.MeasureOverride(IntPtr nativeTarget, Double inWidth, Double inHeight, Double& outWidth, Double& outHeight)
    at MS.Internal.XcpImports.MeasureOverrideNative(IntPtr element, Single inWidth, Single inHeight, Single& outWidth, Single& outHeight)
    at MS.Internal.XcpImports.FrameworkElement_MeasureOverride(FrameworkElement element, Size availableSize)
    at System.Windows.FrameworkElement.MeasureOverride(Size availableSize)
    at System.Windows.Controls.ScrollContentPresenter.MeasureOverride(Size constraint)
    at System.Windows.FrameworkElement.MeasureOverride(IntPtr nativeTarget, Double inWidth, Double inHeight, Double& outWidth, Double& outHeight)
    at MS.Internal.XcpImports.MeasureNative(IntPtr element, Single inWidth, Single inHeight)
    at MS.Internal.XcpImports.UIElement_Measure(UIElement element, Size availableSize)
    at System.Windows.UIElement.Measure(Size availableSize)
    at System.Windows.Controls.ScrollViewer.MeasureOverride(Size constraint)
    at System.Windows.FrameworkElement.MeasureOverride(IntPtr nativeTarget, Double inWidth, Double inHeight, Double& outWidth, Double& outHeight)
    at MS.Internal.XcpImports.MeasureOverrideNative(IntPtr element, Single inWidth, Single inHeight, Single& outWidth, Single& outHeight)
    at MS.Internal.XcpImports.FrameworkElement_MeasureOverride(FrameworkElement element, Size availableSize)
    at System.Windows.FrameworkElement.MeasureOverride(Size availableSize)
    at System.Windows.FrameworkElement.MeasureOverride(IntPtr nativeTarget, Double inWidth, Double inHeight, Double& outWidth, Double& outHeight)
    at MS.Internal.XcpImports.MeasureOverrideNative(IntPtr element, Single inWidth, Single inHeight, Single& outWidth, Single& outHeight)
    at MS.Internal.XcpImports.FrameworkElement_MeasureOverride(FrameworkElement element, Size availableSize)
    at System.Windows.FrameworkElement.MeasureOverride(Size availableSize)
    at System.Windows.FrameworkElement.MeasureOverride(IntPtr nativeTarget, Double inWidth, Double inHeight, Double& outWidth, Double& outHeight)

On a hunch I created non-generic version of my converter and it seemed to work. Oh, so I can't use generic base for UI controls? Nice job MS.

public class EqualToVisibilityConverter : FrameworkElement, IValueConverter
{
    public static readonly DependencyProperty ParameterProperty = DependencyProperty.RegisterAttached(
        "BindingParameter",
        typeof(int),
        typeof(EqualToVisibilityConverter),
        new PropertyMetadata(0));

    private static void OnBindingParameterChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
    {
    }

    public int BindingParameter
    {
        get { return (int)GetValue(ParameterProperty); }
        set { SetValue(ParameterProperty, value); }
    }

    public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
    {
        int bp = BindingParameter;
        return (value.Equals(BindingParameter)) ? Visibility.Visible : Visibility.Collapsed;
    }

    public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
    {
        throw new NotImplementedException();
    }
}

Note that I wrote "seemed to work". It means only that the code didn't explode in my face, but it didn't work quite as I expected. Here's the quick breakdown: when the visibility converter is called for the first time, the bound parameter is null. Then the dependency value is changed (at least the callback is called; curiously, the setter is not) and the bound parameter is initialized with a proper value, but there is no way to invalidate the conversion result.

That gave me another idea: why use IValueConverter interface at all? Let's create an UI element with 2 dependency properties for arguments and one property for result of binary operation:

public class EqualToVisibilityConverter : FrameworkElement, INotifyPropertyChanged
{
    public static readonly DependencyProperty LeftArgumentProperty = DependencyProperty.Register(
        "LeftArgument",
        typeof(int),
        typeof(EqualToVisibilityConverter),
        new PropertyMetadata(OnArgumentChanged)
        );

    public static readonly DependencyProperty RightArgumentProperty = DependencyProperty.Register(
        "RightArgument",
        typeof(Visibility),
        typeof(EqualToVisibilityConverter),
        new PropertyMetadata(OnArgumentChanged)
        );

    private static void OnArgumentChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
    {
        EqualToVisibilityConverter converter = d as EqualToVisibilityConverter;
        converter.Result = (converter.LeftArgument == converter.RightArgument) ? Visibility.Visible : Visibility.Collapsed;
    }

    public int LeftArgument
    {
        get { return (int)GetValue(LeftArgumentProperty); }
        set { SetValue(LeftArgumentProperty, value); }
    }

    public int RightArgument
    {
        get { return (int)GetValue(RightArgumentProperty); }
        set { SetValue(RightArgumentProperty, value); }
    }

    private Visibility _result;
    public Visibility Result
    {
        get { return _result; }
        set
        {
            if (value != _result)
            {
                _result = value;
                OnPropertyChanged("Result");
            }
        }
    }

    public event PropertyChangedEventHandler PropertyChanged;
    protected void OnPropertyChanged(string propertyName)
    {
        if (null != PropertyChanged)
        {
            PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
        }
    }
}

Now the callbacks of dependency properties can trigger the result recalculation and everything will be fine and dandy. Edit XAML, rebuild, run, KABOOM! This time application blew up, but I didn't even get a call stack, it just closed without any message.

This. Is. Fucking. Ridiculous.

At this moment I decided I already lost enough time on this crap, so I just said "screw it" and added additional property to OpenListSetting class.

This is when I finally got why the MVVM pattern got so much traction with XAML developers. I always assumed that it's supposed to be used for big model adaptations, and all minor stuff should be done in binding converters. However it seems that you have to use ViewModel for anything remotely useful. Don't get me wrong: I agree that preventing pollution of Model by stuff needed by View is a Good Thing TM, but I think a separation of View into View and ViewModel is necessary only because of limitations of XAML.

MessageBox crash on Windows Phone 7

Guess what's wrong with this Application Bar button handler:

private void OnAboutButtonClick(object sender, EventArgs e)
{
    MessageBox.Show("Blah blah blah, our app v1.0");
}

If you answered "you're using hardcoded string instead of application resource" you are of course right, but that's not the worst problem with it, so no cookie for you. If the user clicks the application bar button twice before the message box is shown it crashes:

System.Exception was unhandled
  Message=0x8000ffff
  StackTrace:
    at MS.Internal.XcpImports.CheckHResult(UInt32 hr)
    at MS.Internal.XcpImports.MessageBox_ShowCore(String messageBoxText, String caption, UInt32 type)
    at System.Windows.MessageBox.ShowCore(String messageBoxText, String caption, MessageBoxButton button)
    at System.Windows.MessageBox.Show(String messageBoxText)
    at PhoneApp1.MainPage.OnAboutButtonClick(Object sender, EventArgs e)
    at Microsoft.Phone.Shell.ApplicationBarItemContainer.FireEventHandler(EventHandler handler, Object sender, EventArgs args)
    at Microsoft.Phone.Shell.ApplicationBarIconButton.ClickEvent()
    at Microsoft.Phone.Shell.ApplicationBarIconButtonContainer.ClickEvent()
    at Microsoft.Phone.Shell.ApplicationBar.OnCommand(UInt32 idCommand)
    at Microsoft.Phone.Shell.Interop.NativeCallbackInteropWrapper.OnCommand(UInt32 idCommand)
    at MS.Internal.XcpImports.MessageBox_ShowCoreNative(IntPtr context, String messageBoxText, String caption, UInt32 type, Int32& result)
    at MS.Internal.XcpImports.MessageBox_ShowCore(String messageBoxText, String caption, UInt32 type)
    at System.Windows.MessageBox.ShowCore(String messageBoxText, String caption, MessageBoxButton button)
    at System.Windows.MessageBox.Show(String messageBoxText)
    at PhoneApp1.MainPage.OnAboutButtonClick(Object sender, EventArgs e)
    at Microsoft.Phone.Shell.ApplicationBarItemContainer.FireEventHandler(EventHandler handler, Object sender, EventArgs args)
    at Microsoft.Phone.Shell.ApplicationBarIconButton.ClickEvent()
    at Microsoft.Phone.Shell.ApplicationBarIconButtonContainer.ClickEvent()
    at Microsoft.Phone.Shell.ApplicationBar.OnCommand(UInt32 idCommand)
    at Microsoft.Phone.Shell.Interop.NativeCallbackInteropWrapper.OnCommand(UInt32 idCommand)

The interesting thing about this stack trace is that it contains two calls to MessageBox.Show(). WTF? Since MessageBox.Show() returns the value, I assumed it's a synchronous call, i.e. no stuff would happen until the user clicks "OK". Apparently MessageBox opens internal event loop (which kind of makes sense, since something has to handle the user clicking "OK"), which handles the second application bar click, tries to open the second message box and the whole application blows up.

Now, you may say "It's only a problem if the user intentionally bangs the poor phone like an ADD afflicted monkey on speed". I say "Fix your god damn app" (and in case anyone from Microsoft would ever read this, I also say "Fix your god damn framework").

So what can we do? Using lock is out of the question, since everything is executed in the same thread. So the only option I came up with is a Crappy Boolean Flag Pattern:

private bool _msgboxShown = false;
private void OnAboutButtonClick(object sender, EventArgs e)
{
    if (!_msgboxShown)
    {
        _msgboxShown = true;
        MessageBox.Show("Blah blah blah, our app v1.0");
        _msgboxShown = false;
    }
}

Not elegant, but it works. Of course if you use message boxes for other stuff, create some helper class for this crap.

Handling QML errors 101

Qt SDK comes with great documentation and examples, but there is hardly any information about handling QML errors. All examples assume that nothing can go wrong during runtime, which is a bit naive approach. Even if your app is completely bug free, the users will find a way to shoot themselves in the foot. They will install your app and then uninstall key plugins ignoring all warnings from system. They will install your app on SD card and then delete some files for some unfathomable reason. And then they will blame you.

Let's consider what happens if you have an error in your main QML file? User sees the black screen, gets pissed off, uninstalls your app and gives it a negative review in Nokia store. Not good. Let's try something different:

int main(int argc, char *argv[])
{
    QApplication app(argc, argv);   
    QDeclarativeView canvas;

    canvas.setSource(QLatin1String("main.qml"));
    if (!canvas.errors().empty()) {
        // handle errors
        QMessageBox msgBox;
        msgBox.setText("Uh oh, something went terribly wrong!");
        msgBox.setInformativeText(
            "We're sorry, but it seems there "
            "are some problems with running "
            "our application on your phone.");
        msgBox.exec();

        return -1;
    }

    // optionally set the screen orientation
    // call show/showFullscreen/showMaximized

    return app.exec();
}

Few notes about the snippet above:

• If you're using the QML Application template from the recent version of Qt Creator you probably have a generated QmlApplicationViewer class instead of raw QDeclarativeView, but the general idea stays the same.
• Error handling section above is only a stub. You may ask user to reinstall the application. You may display more information about the error along with some contact information. If your application uses network connection, you should probably ask user if he wants to send an error report.
• If you use the QMessageBox or other modal dialog and lock the screen orientation, you mustn't lock the orientation before showing the dialog. In current version of Qt (4.7.4) there is a bug and in case of forced orientation change only dialog buttons are displayed.

Updating Qt applications in Nokia Store

Every Symbian Qt application in Nokia Store consists of two parts: metadata and content files. Metadata is the information about the application visible in Nokia Store, and content files are applications binaries, i.e. sis file. Both parts are separate entities, can be updated independently and have separate Quality Assurance process. It stands to reason: why would anyone have to verify the application binary when you correct a typo in applications  description?

But sometimes you might want to change the description and the binary at the same time. For example I screwed up testing of the first version of my "Nu, Pogodi!" game and first few reviews were negative with "Doesn't work, shows only black screen" comment. I fixed the issue, published new binary and added "Version 1.0.1 fixes the black screen issue." text to description. The metadata QA finished earlier, but still the old, malfunctioning binaries were served. Guess what happened - the next review was also negative, this time with "Black screen, cannot download version 1.0.1" text. Joy.

So the point is, currently there is no way to update both parts of content at the same time. The solution recommended by Nokia Publish Support is to publish the binaries first, because the QA for that part is longer than for metadata and update the metadata when binaries pass QA.

x:name binding crash on Windows Phone 7.1

Recently I've started working on the new version of Windows Phone application originally written by interns in my company. One of the first tasks was to migrate from WP 7.0 to 7.1, a.k.a. "Windows Phone 7.5" (because 7.5 sounds sooooo much better than 7.1), a.k.a. "Windows Phone Mango". The transition was generally smooth and it seems we'll be able to clean up some 7.0 specific workarounds, but there was one crash that was quite tricky to track down:

Null Reference Exception:
    at MS.Internal.XcpImports.CheckHResult(UInt32 hr)
    at MS.Internal.XcpImports.UIElement_Measure_WithDesiredSize(UIElement element, Size availableSize)
    at System.Windows.UIElement.Measure_WithDesiredSize(Size availableSize)
    at System.Windows.Controls.VirtualizingStackPanel.MeasureChild(UIElement child, Size layoutSlotSize)
    at System.Windows.Controls.VirtualizingStackPanel.MeasureOverride(Size constraint)
    ...

Our code wasn't even in a call stack! It turns out that the following code worked on 7.0, but crashed on newer version:

<validationcontrol:validationcontrol
    grid.row="1"
    inputscope="Number"
    lostfocus="OnCustomValueChanged"
    text="{Binding Custom}"
    width="420"
    x:name="{Binding Key}"/>

I'm not sure if using binding for a control name and relying on that information in code behind is a good idea, but I'm damn sure it shouldn't crash. And even if it crashes, it should provide some useful information instead of throwing NRE from seemingly random place.

What's the fix for it? Use x:Tag instead of x:Name.

How (not to) test your QML application for Symbian

First of all by QML I do not mean this, I mean this: a UI module of Qt, the cross-platform framework. The gals and guys at Nokia figured out that modern user interface cannot be fully described by static layout in a XML file. Microsoft figured out that too, but they chickened out and only extended XML a bit and added an 'a' to file extension to make it look like something new. Nokians took a step further and created new language for declarative UI based on JavaScript called QML.

The QML UI components can be defined in two ways: it can be a QML file composed of other components (for instance that's the usual way to define the main UI file) or it can be a C++ extension. Both ways can be used together to create a plugin, which can be imported to your project.

And at last we reach the intended topic of this post: testing. What happens if some QML file defining a component is missing? What happens if the whole plugin is missing or the version of this plugin is lower than the one required by application? QML files are interpreted during runtime, so of course you get the runtime error. In the best scenario it limits the functionality of your app, in the worst case it renders it completely unusable.

But hey, you can catch most of those errors simply by clicking through your application, right? Not exactly, doing so only tells you that in works on one particular device. You might have some plugins already installed, but not included in application's package and your app will work only on the devices which happen to have those plugins, which is not very likely.

That's exactly the error I made when I published the first version of "Nu, Pogodi!". I submitted for Q&A process an application with dependencies to Qt Components 1.1, build with the latest Qt SDK. I've tested it thoroughly on some devices I had access to and via Remote Device Access service (which BTW rocks; I wish there was a similar service for Android) and everything worked fine. The application was rejected by Q&A, because at the end of 2011 there was some technical issues with Nokia Store and latest Qt and I was told to rebuild my application with old SDK, which included only Qt Components 1.0. I've tested my game again and everything worked so I published it to Nokia Store. Few days later I received first reviews - all negative, along the lines "doesn't work, beware".

Qt Smart Installer partially prevents those errors, but you still might shoot yourself in the foot in some cases. My game had dependencies to Qt Components 1.1, but the pkg file declared dependency to version 1.0, because it was created with old SDK. When my customers installed the game, the smart installer ensured only that version 1.0 is installed, but my game needed newer version and failed during runtime. I didn't caught this during testing, because all of my devices had latest Qt Components installed.

That was the "How not to test your QML application" part, now let's get to solution. It's really simple: downgrade all the stuff needed by your application to versions defined in pkg file. To check the current versions of Qt libraries and plugins I recommend using an excellent QtInfo tool. To downgrade Qt you need the sis files distributed with old Qt SDK versions.

This simple steps should ensure that your application will work properly on all supported devices. Nevertheless, you should prepare for failure and handle all runtime errors in a user friendly way. But that's the topic for another post...