Category Archives: CMake

FOSDEM CrossDesktop DevRoom 2013 – Call for Talks

FOSDEM is one of the largest gatherings of Free Software contributorsin the world and happens each February in Brussels (Belgium). One ofthe tracks will be the CrossDesktop DevRoom, which will hostDesktop-related talks.

We are now inviting proposals for talks about Free/Libre/Open-sourceSoftware on the topics of Desktop development, Desktop applicationsand interoperativity amongst Desktop Environments. This is a uniqueopportunity to show novel ideas and developments to a wide technicalaudience.

Topics accepted include, but are not limited to: Enlightenment, Gnome,KDE, Unity, XFCE, Windows, Mac OS X, general desktop matters,applications that enhance desktops and web (when related to desktop).

Talks can be very specific, such as developing mobile applicationswith Qt Quick; or as general as predictions for the fusion of Desktopand web in 5 years time. Topics that are of interest to the users anddevelopers of all desktop environments are especially welcome. TheFOSDEM 2012 schedule might give you some inspiration:
https://archive.fosdem.org/2012/schedule/track/crossdesktop_devroom.html

Please include the following information when submitting a proposal:

  • Your name
  • The title of your talk (please be descriptive, as titles will belisted with around 250 from other projects)
  • Short abstract of one or two paragraphs
  • Short bio
  • Requested time: from 15 to 45 minutes. Normal duration is 30minutes. Longer duration requests must be properly justified.

The deadline for submissions is December 14th 2012. FOSDEM will beheld on the weekend of 2-3 February 2013. Please submit your proposalstocrossdesktop-devroom@lists.fosdem.org(subscribtion page for themailing list:https://lists.fosdem.org/listinfo/crossdesktop-devroom)

– The CrossDesktop DevRoom 2013 Organization Team

PS: Qt and KDE people are starting to organize for the booth, devroom, Saturday & Sunday night, etc. If you want to help, join kde-promo and add yourself to the wiki.

 

Guidelines on cross-platform porting

There is a nifty piece of software called zsync, which is kind-of like rsync, except it is totally different.

Rsync

Rsync is mainly useful when you want to synchonize a list of files, or directories, between two servers. It will only download the new files and files which have changed. It will even delete or backup the files which have been removed at the original site. Nice.

For a project I was involved until recently at work we had a slightly different problem: we generate a huge file (an ISO image) which contains about 6 GB of data. This ISO image contains the daily build of our application. It contains only a handful of files. Problem is some of them are generated and GB in size, yet from day to day only maybe 100-150 MB have changed (and it would be even less if it were not because of this “feature” of .NET that never generates identical binaries even if using exactly the same source code)

Rsync was not useful in this case: it would download the whole file, gigabytes! (some of the people downloading the ISO are on a slow link in India)

 

zsync

This is exactly the case zsync targets: zsync will only download the changed parts of the file thanks to the rolling checksum algorithm.

Best of all: no need for an rsync server, opening port TCP 873 (which requires months of arguing with BOFHs in some companies), or anything special: HTTP over port 80 and you are done. Provided that you are not using Internet Information Server, which happens to support only 6 ranges in an HTTP request (hint: configure nginx in reserve proxy mode).

But I’m digressing.

Cool. Great. Awesome. Zsync. The perfect tool for the problem.

 

Hello Windows

Except for this project is for Windows, people work on Windows, they are horrified of anything non-Windows, and zsync is only available for Unix platforms.

Uh oh.

In addition to that, the Cygwinport suffers from many connection error problems on Windows 7 and does not work on a cmd.exe prompt, it wants the Cygwin bourne shell prompt.

So I started to port zsync to Windows natively.

 

Native port howto

The starting point was:

  • C99 code
  • autotools build system
  • No external dependencies (totally self-contained)
  • Heavy use of POSIX and Unix-only features (such as reading from a socket via file descriptors, renaming a file while open, deleting a file while open and replacing it with another file yet still use the same file descriptor, etc)

To avoid breaking too much, and because I wanted to contribute my changes upstream, my intention was to do the port step by step:

  1. Linux/gcc/autotools
  2. Linux/gcc/CMake
  3. Cygwin/gcc/CMake
  4. MSYS/MinGW gcc/CMake
  5. Visual C++/CMake

 

Autotools

Autotools was the first stone in the path.

With some work (calling MSYS from a DOS prompt, etc) it would have been possible to make it generate a Visual C++ Makefile but it would have been painful.

Plus the existing autotools build system did not detect the right configuration on MinGW.

Step 1: replace autotools with CMake. On Linux. This was relatively easy (although time consuming) and did not require any change in the code.

 

Cygwin

The second step was to build zsync on Windows using Cygwin (which provides a POSIX compatibility layer) and CMake.

No code changes were required here either, only a few small adjustments to the CMake build system. I tested on Linux again, it worked fine.

At this point, I had only made a pyrrhic progress: zsync was still Unix only, but with a cross-platform build system.

 

MinGW

My next step was a serious one: port zsync to use MinGW, which generates a native Windows application with gcc.

That means using Winsock where required.

5And hitting Microsoft’s understanding of “POSIX-compliant”: the standard Windows POSIX C functions do not allow to treat sockets as files, rename open files, temporary files are created in C:\ (which fails on Windows Vista and newer), etc. And that’s when the functions do exist. In many cases (mkstemp, pread, gmtime_r…) those functions were outright inexistent and I needed to provide an implementation.

Plus adapting the build system. Fortunately, I was still using gcc and Qt Creator provides great support for MinGW and gdb on Windows, and decent support for CMake.

Some other “surprises” were large file support, a stupid “bug”and the difficulties of emulating all the file locking features of Unix on Windows.

Regarding LFS, I took the easy path: instead of using 64-bit Windows API directly, I used the mingw-w64 flavor of gcc on Windows, which implements 64-bit off_t on 32-bit platforms transparently via _FILE_OFFSET_BITS.

 

Visual C++ misery

Porting to Visual C++ was the last step.

This was not strictly required. After all, all I had been asked for as a native version, not a native version that used Visual C++.

Yet I decided to give VC++2010 a try.

The main problem was lack of C99 support (though you can partially workaround that by compiling as C++) and importing symbols due to lack of symbol exports in the shared library (attributes for symbol visibility were introduced in gcc4.0, but many libraries do not use them because gcc does its “magic”, especially MinGW, which will “guess” the symbols).

Porting to Visual C++ 2010 required either to give up some C99 features in use (e. g. moving variable declarations to the beginning of the functions) or adding a lot of C++-specific workarounds (extern “C”).

I was a bit worried upstream would not accept this code because it didn’t really provide any benefit for the application (for the developer, use of a great IDE and very powerful debugger), therefore I didn’t finish the Visual C++ port. Maybe some day if Microsoft decides to finally provide C99.

The result (so far) is available in the zsync-windows space in Assembla.

 

Reminder: FOSDEM 2012 CrossDesktop DevRoom Call for Talks

FOSDEM is one of the largest gatherings of Free Software contributors in the world and happens each February in Brussels (Belgium). One of the developer rooms will be the CrossDesktop DevRoom, which will host Desktop-related talks.

Are you interested in giving a talk about open source and Qt, KDE, Enlightenment, Gnome, XFCE, Windows, Mac OS X, general desktop matters, mobile development, applications that enhance desktops and/or web?

Hurry up and submit your proposal, deadline is December 20th!

There is more information in the Call for Talks we published one month ago.

If you are interested in Qt/KDE, come visit us at the KDE booth. If you add yourself to the KDE FOSDEM 2012 wiki page, we will be able to better organize the usual dinner on Sunday and/or smaller meetings for “special interest groups”.

 

FOSDEM 2012 – CrossDesktop DevRoom Call for Talks

FOSDEM is one of the largest gatherings of Free Software contributors in the world and happens each February in Brussels (Belgium). One of the developer rooms will be the CrossDesktop DevRoom, which will host Desktop-related talks.

We are now inviting proposals for talks about Free/Libre/Open-source Software on the topics of Desktop development, Desktop applications and interoperativity amongst Desktop Environments. This is a unique opportunity to show novel ideas and developments to a wide technical audience.

Topics accepted include, but are not limited to: Enlightenment, Gnome, KDE, XFCE, Windows, Mac OS X, general desktop matters, applications that enhance desktops and web (when related to desktop).

Talks can be very specific, such as developing mobile applications with Qt Quick; or as general as predictions for the fusion of Desktop and web in 5 years time. Topics that are of interest to the users and developers of all desktop environments are especially welcome. The FOSDEM 2011 schedule might give you some inspiration.

Please include the following information when submitting a proposal: your name, the title of your talk (please be descriptive, as titles will be listed with around 250 from other projects) and a short abstract of one or two paragraphs.

The deadline for submissions is December 20th 2011. FOSDEM will be held on the weekend of 4-5 February 2012. Please submit your proposals to crossdesktop-devroom@lists.fosdem.org

Also, if you are attending FOSDEM 2012, please add yourself to the KDE community wiki page so that we organize better. We need volunteers for the booth!

 

A wish a day 11: Perl Compatible Regular Expressions in CMake

CMake is a unified, cross-platform, open-source build system that enables developers to build, test and package software by specifying build parameters in simple, portable text files. It works in a compiler-independent manner and the build process works in conjunction with native build environments, such as Make, Apple‘s Xcode and Microsoft Visual Studio. It also has minimal dependencies, C++ only. CMake is open source software and is developed by Kitware.(taken from Wikipedia)

But you already know that because CMake is the build system we use in KDE :-)

When I started to use CMake at work to port a large project from Windows to Linux, I needed to write a lot of modules (“finders”). Those modules had to work on Windows and Linux, and sometimes with different versions of some of the third-party dependencies. Parsing header files, Windows registry entries, etc was often required.

If you have ever parsed text, you know you powerful regular expressions are. If you use old software, you know how great Perl Compatible Regular Expressions (PCRE) are because you do not have them available :-).

Continue reading

A wish a day 7: make emerge a generic package manager for Windows

A while ago I said Koen from Emweb made an interesting proposal at FOSDEM about emerge, the KDE Windows build tool.

Yesterday, Jarosław Staniek and I reaffirmed our commitment to ‘emerge’. Today, I’d like to go a bit further: let’s bring more developers to emerge by opening it up to other projects. Keep reading!

What is emerge, why is it important and what was Koen’s proposal?

Fact: Microsoft Windows is very different to Unix in regards to development.

On Unix platforms -that includes Linux and Mac OS X-, software is usually installed to /usr: applications in /usr/bin and /usr/sbin, libraries in /usr/lib, headers in /usr/include, common resources in /usr/share, etc. Also, dependency management is usually something you can count on: when you install kdelibs5-dev in Ubuntu, it will automatically install libqt4-dev, kdelibs5-data, libfreetype (runtime), etc That makes setting up a development environment a very easy task: look for shared libraries, header files, etc in the common places and you will probably find them.

On Windows there is nothing like that. When you want to compile an application, you need to provide (build and install) all its dependencies, and you need to tell Visual Studio where to find everything. Even CMake usually needs some help in the form of a hint for CMAKE_PREFIX_PATH. As you may imagine, building KDE, which has more than 200 third party dependencies and tens of modules (and with the move + split to git, many more) becomes an almost insurmountable task.

‘Emerge’ to the rescue: inspired by Gentoo‘s emerge, Ralf Habacker, Christian Ehrlicher, Patrick Spendrin and others (yours faithfully included) developed a tool which downloads the source, configures, builds, installs and packages KDE and its dependencies. It makes a world of difference when building KDE. Actually, it makes building KDE on Windows possible. Once more: thank you very much guys, impressive tool.

There are two well-differentiated parts in emerge, the ‘engine’ and the ‘recipes’.

Continue reading

Bootstrap your CMake project

When I talk to people about the CMake build system, they like the fact that it generates native project files for the tools you already use (which makes possible to use distributed compiling, debugging, etc). Next to that, Visual Studio and autotools users complain about bootstrapping not being possible.

If you are using Visual C++, solutions probably work fine for you and it is a fact they work out of the box with VC++. However, do not even try to move to another platform (Linux, Mac…) or another compiler on Windows because it will not work.

If you use autotools, you generate the configure script on your development machine and your users just run it to prepare makefiles. Only a Bourne-compatible shell is required when trying to build. Moving to Windows, particularly Visual C++ will be quite hard.

CMake is in my opinion one of the best build-systems out there but there is one problem: you need to have CMake in order to build a CMake-based project.

Now add third-party dependencies, which probably use a different build-system, to the mix and suddenly building your software has become time-consuming and complex. In terms of money, that translates to lost revenue because there is a high probability people evaluating your software will give up before they even have something to test.

Nokia solves this by distributing Qt binaries for the most popular compilers (MSVC2005, MSVC2008, MinGW). But there is always someone using a different compiler: MSVC2010, a different flavor of MinGW, Embarcadero C++ Builder

KDE on Windows solves this by having its own “meta-buildsystem” called emerge, which is written in Python, which adds a dependency on Python when you want to build.

Can’t we do better? Can we get all the advantages of CMake yet not depend on CMake being available?

Yes, we can. I tried to do that for Wt and implemented it in the form of winstng. It’s a small batch file for Windows and a Bash shell script (for Unix; I didn’t manage to remove one bashism) which automagically download, build (if needed) and install CMake, all the third-party dependencies, build them, and then download Wt and install it. Everything ends up in a self-contained location which you can move around*. And you do not need administrator permissions. Tested on Windows, Linux and Mac.

*For now that’s only true on Windows, on Unix platforms I have not removed the rpath for some libraries which use build systems other than CMake (namely, OpenSSL).

BTW, if you are at FOSDEM, there is a Wt talk on Sunday, 14.30-15.15: The next desktop is the browser.

KDE for Haiku

Haiku is a free open source operating system compatible with BeOS. I’d say the BeOS community is the new Amiga community: they love their platform and will endure as many difficulties as necessary as long as they are able to run BeOS/Haiku.

A few months ago, Qt4 was ported to BeOS/Haiku. That’s great news because it means we Qt4 developers can now target a new platform (did I say CMake is also available for Haiku?). Arora quickly followed.

Now, KDE 4 is also available for Haiku. Freezing cool.

By the way, OS/2 is another of those everybody-thought-they-were-dead platforms which are being infused new software thanks to ports of Qt4 (they need money to complete the port, donate!) and CMake

Please note I am not a KDE-on-Haiku or KDE-on-OS/2 developer. I know zero of Haiku/BeOS and I forgot almost all I knew of OS/2 programming.

PS: Comments here do not work due to hosting manfunction

ANN: CModuler 1.0

CModuler is a CMake module generator. I was fed up with copy & paste, search & replace over and over again 90% of the time.

Version 1.0 (AKA “CModuler Meta”) provides only very limited functionality: it will create finders (modules of the form FindXXX.cmake) for libraries consisting of only 1 library. CModuler-generated modules support debug/release configurations.

Here is an example of a CModuler-generated module for finding Sqlite3: FindSqlite3.cmake

You can download CModuler from Gitorious

Like CMake, CModuler is licensed under the BSD license

NB: The comments in this blog do not work due to a hosting issue