Thursday, May 03, 2012

Building a Build Process, Part 8: CruiseControl.NET Preparation

This is part of a larger series in building a proper build process. For more information and additional series links, check out the introductory post.

Choosing Whether to Run as Network Service or another user

This was brought up as an excellent point by Bahrep on a recent blog post of mine, and I thought it was worth sharing for this series.

Essentially, the issue is that CruiseControl.NET runs under the network service user (which makes sense), but in order to accept an SVN certificate, you need to be logged in as that user and check out a copy of the repository once to accept the certificate before CruiseControl will be able to connect to the repository as itself.

There are two ways you can go about this. The more visible way is to create a local user account on the server (i.e. “CI”) which you then tell CruiseControl to run as. You give CI the permissions it needs, including signing on to that user account once , checking out the repository, and accepting the certificate.

The second option is to download and install PsExec, which lets you run programs under other user accounts. You then use PsExec to run the checkout command as the network service user and accept the certificate. No extra user account, no changing how the service runs. It’s a simpler solution, but a much less visible change that you’ll want to document.

Directions for both options are shown below. For what it’s worth, I now prefer option 2.

Option 1 Part 1: Creating a Local User for CI Purposes

  • Click start and type “compmgmt.msc” to bring up Computer Management.
  • In the left-hand navigation, choose Local Users and Groups –> Users.
  • Right-click and select New User…
  • Name the user “CI” or something related to the purpose at hand.
  • Give the user a password.
  • Uncheck “User must change password”
  • Check “password never expires”.
  • Create the User account.
  • Right-click the new account and select Properties.
  • Click the “Member Of” tab.
  • Add the user to the “Administrators” Group. NOTE: This is not ideal for the purposes of our demo. Normally, you’d want to ensure this user account only has access to what it needs for CC.NET (this is why I’m leaning towards option 2).
  • Now is also a good time to install TortoiseSVN on the build server, if you haven’t yet. (be sure to install the command-line client tools as well!)

Option 1 Part 2: Accepting the SVN Certificate as the CI User

  • Log off the Administrator account and sign in as your new CI account.
  • Run the command prompt
  • Run svn info to accept the certificate. This can be done by running svn info https://[repo URL or hostname]svn/TestProjectRepo –username svnuser1 –password passw0rd1
  • Type “p” to accept the certificate permanently, and you’re all set!

Option 1 Part 3: Run the CruiseControl.NET Service as the CI User

  • Log off the CI account and back into the Administrator account.
  • Run services.msc
  • Right-click the CruiseControl.net service and choose Properties
  • Click the Log On tab.
  • Click the “This account” radio button.
  • Click Browse and Type CI. THe resulting username entry will be “.\CI”
  • Enter the password you created for the CI account and click apply.
  • You will see a message that it has been granted the “Log On as a Service’ Right.

Option 2: Using PsExec to Accept the SVN Cert as the Network Service User

NOTE: At points during this process, you may receive a warning from your antivirus program. This is because PsExec can be dangerous when misused, and viruses have used it in attacks in the past. We know our usage is legitimate here, so we can unblock/ok PsExec operations when we’re working with it.
  • Visit the PsExec Website and Download the application (actually will be a zip file of all the PSExec tools)
  • Unzip the PsExec zip file
  • open a command prompt and navigate to the folder of PsTools
  • Run psexec -u "nt authority\network service" cmd.exe. This will run the command prompt as the network service user.
  • Run svn info to accept the certificate. This can be done by running svn info https://[svn ip or hostname]/svn/TestProjectRepo –username svnuser1 –password passw0rd1
  • Type “p” to accept the certificate permanently, and you’re all set!

Add a CI User to the Repository

  • Open your CentOS source code management VM
  • Taking advice from our Subversion & Apache article, we’ll run the following command to add a new CI user without deleting the old ones: htpasswd -sb /var/www/svn/auth/svn.htpasswd ci passw0rdci
Now you’re set to allow CruisControl to pull down files under its own again (just around the bend in this series when we configure CCNet).

Copying the Microsoft Targets to the Build Server

When dealing with this elsewhere, I discovered this problem. We’ll avoid it in advance here.
  • You have to copy the two files from your local development machine (with VS installed) to the Build Server.
  • Copy the directory C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\*.* to the same location on the build server.

Copying the Reference Assemblies to the Build Server

When dealing with this elsewhere, I discovered this problem. We’ll avoid it in advance here.
  • On your local development machine, copy C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\*.* to a folder on your build server. For me, it was E:\ContinuousIntegration\_ReferenceAssemblies so that it could keep it common for any future builds.
In the next part in the series, we’ll finally configure CruiseControl.NET and get it to build. Thanks for hanging out with me on this epic journey towards build awesomeness!

Feedback Welcome!

I'd love to hear any comments on this series. Find it useful? Think there's a better way to implement the technique or something I should have mentioned? Please drop a line in the comments to help me improve the series!

References


<—Part 7: Installing CruiseControl.NET
Part 9: coming soon! -->

Building a Build Process, Part 7: Installing CruiseControl.NET

This is part of a larger series in building a proper build process. For more information and additional series links, check out the introductory post.

Adding a HDD to your VM for CI Information

I find this makes it easier to keep your CI files separate, and it’s definitely a good practice in a production environment in my experience. Virtualization makes this easier as well because all the virtual hard drive files only take up as much space as they use. Definitely an upside to VMs.
To add a new HDD to the VM, do the following:
  • Power down your Windows Build Server VM if it’s on.
  • Open VirtualBox
  • Right-click on your BobTheBuilder machine and select “Settings…”
  • Click the “Storage” Section on the Left-hand side
  • Note the two icons next to the “SATA Controller” section. Click the one on the right (“Add Hard Disk”). The Hard disk wizard will open.
  • Click “Create New Disk”
  • Choose your Disk Format (I left it as the default, VDI)
  • Choose “Dynamically Sized”
  • Name the Disk. I usually follow the format of [MachineName]_[DrivePurpose], so in this case I chose “BobTheBuilder_CIDrive”
  • Set the size of the drive. I left the default (25 GB)
  • Next screen is the summary. Click create to create the drive.
  • Start the VM again and login.
Now we have a hard drive initialized, but we still need to format it before the OS can see it. To do this:
  • Click start and type “diskmgmt.msc” to bring up the disk management utility.
  • You’ll see a dialog box to ask you to initialize the disk.
  • Select MBR for the type of initialization and click OK.
  • Now, right-click on the “Disk 1” entry to the bottom (the disk should indicate that it has 25 GB of unallocated space) and select “New Simple Volume”, which will open a wizard.
  • The amount of space for the drive will default to 100%. This is what we want, so click next.
  • Assign the drive letter of your choice. The default was E: for me, so I left it. Henceforth in this tutorial series, I’ll be calling it E:\ so you may want to choose E: to make it easier to follow along.
  • On the next screen, choose to format the volume as NTFS and give it the volume label of “CI”.
  • Click Finish to complete the process.
After a few seconds, you should see the E: drive appear in your drives list. Open Windows Explorer and verify that it exists there, too.

Creating a Home for Our CI Files

Now that we have a drive, it makes sense to create our directory structure.
On the E:\ drive, create the following directory structure:
  • TestProject\
    • CIArtifacts\
    • WorkingDirectory\
CIArtifacts will store the output of our CI (logs, etc.); WorkingDirectory will be where we eventually check out the source code to automatically in order to act upon it.

Downloading the CruiseControl.NET Binaries


Installing CruiseControl .NET

  • Run the CruiseControl.NET Setup executable as an administrator by right-clicking and selecting “Run as Administrator”.
  • Agree to the license agreement.
  • All options are selected by default; leave them and continue.
  • Leave both checkmarks checked to install the CruiseControl .NET dashboard and to install CCNet as a service.
  • Leave the default installation directory or customize to your preference (I left the default for this setup)
  • Leave the default value for the Program Files folder group and click Next. CruiseControl.NET will commence installation.
  • Click Finish and exit the wizard.

Installing CCTray

CCTray allows you to connect to one or more CruiseControl.NET projects and will keep you informed on their status.
You’ll want to repeat this process on the Build Server itself and on any desktops you’d like to see the status of the build on (for example, I have CCTray installed on my laptop’s desktop so I can quickly see if a build is broken).
To install CCTray, perform the following (don’t worry, we’ll configure it later):
  • Run the CCTray setup file and click Next at the introduction.
  • Agree to the license agreement.
  • Leave all three options selected and click next.
  • Click next through the installation location and start menu group name screens. The application will install.
  • Click next, and then click Finish, leaving the checkbox selected to start the program.
Now CCTray is started (though not configured yet.)

Install / Start the CCNet Dashboard in IIS

  • Click Start –> Administrative Tools –> IIS Manager
  • Expand the tree on the left-hand side to [Server Name] –> Sites –> Default Web Site.
If you don’t see a directory under “Default Web Site” called “ccnet”, perform the following steps (otherwise, skip to after this bulleted list):
  • Right-click on Default Web Site and select “Add Virtual Directory…”
  • Give “ccnet” as the alias
  • For the path, choose [CCNet Install Directory]\webdashboard. (e.g. for me, it was C:\Program Files (x86)\CruiseControl.NET\webdashboard)
  • Click “OK”. The Virtual Directory will appear.
  • Right-click the ccnet Virtual directory and choose “Convert to Application”.
  • Click OK in the dialog box that appears.
Now your ccnet virtual directory is set up. Keep going:
  • Click on “Default Web Site”
  • On the right-hand side Action menu, click “Start” to start the default web site (if it’s not already started).
  • Open a web browser on the build server and navigate to http://localhost/ccnet
If a web site shows up at all, you’re good to go. Don’t worry about any errors within the web dashboard application, as we’ll be configuring CCNet later.


Feedback Welcome!


I'd love to hear any comments on this series. Find it useful? Think there's a better way to implement the technique or something I should have mentioned? Please drop a line in the comments to help me improve the series!

References


<—Part 6b: MSBuild Integration With Cassini and Visual Studio

Wednesday, May 02, 2012

Building a Build Process, Part 6b: MSBuild Integration With Cassini and Visual Studio

This is part of a larger series in building a proper build process. For more information and additional series links, check out the introductory post.

This Time…

In this round, we’re going to discuss:
  • How to Start the Cassini Web Server Asynchronously
  • How to Stop the Cassini Web Server
  • How to (not quite) get Visual Studio to seemlessly Follow the Same directions that your MSBuild file follows.

Starting the Web Server (Asynchronously!)

Visual Studio has a built-in web server – no doubt you’re familiar with it. It’s what runs whenever you hit F5 on a web server and see a web site come up. This server is called Cassini, and you can start it up through an MS Build Task.
[A little background: Rather than show here, I’m going to tell. You can use the Exec task normally to run an executable, but the catch is that MSBuild will usually wait for the task to finish. We’re going to use the AsyncExec task in order to ensure that MSBuild will start the web server and continue performing next commands without waiting for Cassini to exit, since waiting on Cassini would be undesirable behavior here.]
We’re going to take advantage of an excellent set of extensions called the MSBuild Community Tasks to accomplish our mission here.

Get the MSBuild Extension Pack

  • You should visit the MSBuild Extension Pack web site for an excellent overview of the capabilities of these tools. You can just click the download button on the right-hand side to get the latest version in ZIP format..
  • Unzip the download (anywhere is fine).
  • Go one level deeper and extract the .NET 4.0 zip file.
  • Create a folder in the “thirdparty\tools” folder of your solution called “MSBuildExtensionPack”.
  • Copy the contents of the “Build” folder from the zip file (the .dlls, etc.) into this directory.

Updating Your Build File to be AsyncExec Ready

We’ll have to update the build file to bring in the new task library (a great feature of MSBuild, by the way). To do this, we’ll add an import directive to the Extension Pack’s Task Files (this should go just inside of the <project> tag):
<Import Project=".\thirdparty\tools\MSBuildExtensionPack\MSBuild.ExtensionPack.tasks"/>         

Next up, we have to make a slight modification. The Extension Pack attempts to do some nice work for us to include all the tasks, but we need to override it. Find the following section at the top of the MSBuild.ExtensionPack.Tasks file and comment it out (in the lines below, I’ve done that already for you):


<!--   
<PropertyGroup>
        <BuildPath Condition="'$(BuildPath)' == ''">$(MSBuildProjectDirectory)</BuildPath>
        <ExtensionTasksPath Condition="Exists('$(BuildPath)\..\..\BuildBinaries\MSBuild.ExtensionPack.dll')">$(BuildPath)\..\..\BuildBinaries\</ExtensionTasksPath>
        <ExtensionTasksPath Condition="'$(ExtensionTasksPath)' == ''">$(MSBuildExtensionsPath)\ExtensionPack\4.0\</ExtensionTasksPath>
    </PropertyGroup>
-->

Since the Extension Pack is no longer figuring out what its path is, we need to set an item in our <ItemGroup> to point it to the right place:

        <ExtensionTasksPath Include=".\thirdparty\tools\MSBuildExtensionPack\"/>

Now we’re ready to add the command to start the web server.



Adding a Target to Start the Web Site


First thing’s first – we have to add an item to the <ItemGroup> section to tell MSBuild where Cassini resides, and an item to tell it where our published web site will reside once it’s been spat out by our build process. I took the guesswork out of it for you in the lines below:

<Cassini Include="$(CommonProgramFiles)\microsoft shared\DevServer\10.0\WebDev.WebServer40.exe"/>
<Website Include=".\buildartifacts\_PublishedWebsites\TestProject.Web"/>

Next, we use the AsyncExec task from the Extension Pack to run the web server without hanging up our build process:

    <Target Name="StartWebsite" DependsOnTargets="Compile">         <AsyncExec Command='"@(Cassini)" /port:9999 /path:"%(WebSite.FullPath)" /vpath:'/>     </Target>

Try running your build file with a target of “StartWebsite”. You should be able to navigate to http://localhost:9999/ and see the web site in action (though it may just show a directory’s contents if the site is empty).

However, you may have noticed something. What stops the website so we can start it again? Nothing, and so we’re going to build that target too.



Adding a Target to Stop the Web Site and Updating Dependencies


You can use the handy built-into-windows “TaskKill” program to force a kill of program that has the same name as the WebServer. We just call it with an Exec Command, as shown below:

    <Target Name="StopWebsite">         <Exec Command="taskkill /f /im WebDev.WebServer40.exe" IgnoreExitCode="true" IgnoreStandardErrorWarningFormat="true"/>     </Target>

After that, we add the “StopWebsite” Target as a dependency to StartWebsite (before compile, so we know the site will be down before MS Build erases the files and spits out new ones:

    <Target Name="StartWebsite" DependsOnTargets="StopWebsite;Compile">
...
    </Target>

Go ahead and try running your build with the StopWebsite and StartWebsite targets.

NOTE: You may receive an error upon starting Cassini about there only being one instance allowed on a port. If you do, try changing the port number from ‘9999’ to something else in the StartWebsite task (try to pick a port that’s not being used!).

For reference, at this point, our build file looks like:


Getting Visual Studio to Play Along: Help Needed!


Unfortunately, this is one area that this blog series will fall short. I’ve scoured the internet in an attempt to find out how I can output the \bin and \obj to another folder based on the $(SolutionDir) variable, but apparently unlike C++, Visual Studio for C# does not allow this and instead creates a strange Folder with “$(SolutionDir)” literally in the name. I thought it would be pretty straightforward, but boy was I wrong. If anyone has any suggestions, I’m all ears. I was told I could go the route of editing the .csproj file, but I really tend to be wary of that kind of text editing; I like Visual Studio to be able to own that file for its sake.

For now, I just recommend using TortoiseSVN to ignore those folders in your source control so that it doesn’t conflict with anyone else if you hit F5 and commit later.

And by all means, if you know how to solve the mystery, sound off in the comments!

Feedback Welcome!


I'd love to hear any comments on this series. Find it useful? Think there's a better way to implement the technique or something I should have mentioned? Please drop a line in the comments to help me improve the series!



References









<— Part 6: Creating a Custom MS Build File

Tuesday, May 01, 2012

Workaround: MSBuild Error MSB3644: "The reference assemblies for framework ".NETFramework,Version=v4.0" were not found" [Field Notes]

Problem

When attempting to build a Continuous Integration solution with MSBuild on Windows Server 2008 R2 (With .NET Framework 4.0 installed) I receive the following error:


c:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(847,9): warning MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.0" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend.


Solution

I'll be the first to admit there are probably more elegant ways, but I recommend the following:

Install .NET Framework 4.0 / Windows 7 SDK
You can download it from Microsoft. If you don't have access to do this, or if that doesn't work, you can do the following.

Copy the Reference Assemblies Folder to Your Build Server
Copy C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\*.* to a folder on your build server (for me, it was E:\ContinuousIntegration\_ReferenceAssemblies)

Override the Reference Assemblies in Your Build Configuration
When you call MSBuild, pass it a parameter -p:FrameworkOverride, with the location of the build.

For example, my CruiseControl.NET config MSBuild section for this project now looks like:



It's hack, for sure, but it got the solution to build, and it's reusable in future scenarios.

Probably easier to download the SDK and install it, though.

CruiseControl .NET Gotcha: Moving Microsoft.WebApplications.Targets to the server [Field Notes]

Problem

When attempting an automated build, CruiseControl.NET (running on Server 2008 R2 with .NET Framework 4.0 Installed) gives the following error in its log:


error MSB4019: The imported project "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.


Solution

This is ridiculous, but you have to copy the two files from your machine to the Build Server.

Copy the directory C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\*.* to the same location on the build server.

CruiseControl .NET, VisualSVN, and SSL Certificates [Field Notes]

This is a quick post for my reference. Let me know if details would be helpful and I'll be happy to turn it into more of a tutorial style.

Problem

I am integrating CruiseControl.NET with VisualSVN. I'm using a self-signed https on VisualSVN that doesn't match. Because I can't get CruiseControl to accept the certificate permanently, I can't get it to check out files.

Solution


  • You should have a local user account for your build process (with only access to what it needs, of course). This is essentially a local service account.
  • Log on to that service account on the local machine.
  • Using the command line, check out the VisualSVN repository into a folder you created and accept the certificate. Something along the lines of "svn.exe checkout https://[servername]:8443/svn/[ProjectName] --username [user] --password [password]" should do it.
  • The certificate message will then pop up. Type "p" to accept it permanently.
  • Now you have an account that has the access you need.
  • Go into services.msc 
  • Set the CruiseControl.NET service to run as the local build user service account, with the password.
  • Restart the CruiseControl.NET
  • It now should have access and acceptance of the certificate.

Monday, April 30, 2012

PSA: IBM Maximo 7.1 User Guide is Now the Product Reference Guide [Field Notes]

Subject says it all.

I was looking for the IBM Maximo 7.1 User's Guide (as there appeared to be one for 6.2.1) and I was having a lot of difficulty finding it, until I found this little ditty on an IBM Resources page:

Looking for the Maximo Asset Management 7.1 User's Guide? It is replaced with the Product Reference Guide.

So there you have it. If you'd like the direct link, you can find the User Guide / Product Reference Guide here.

References