Resetting RavenDB indexes

If you are unlucky, after upgrading your Octopus Deploy server, or after a database restore, you may notice some strange behavior:

• Users, projects or environments might have completely disappeared in the UI
• Trying to navigate may result in constant errors such as "Waited for 15,103ms for the query to return non stale result."

Why do these happen? Octopus is built on top of RavenDB, a document database. Data in Raven is persisted to ESENT, the same database technology that powers Microsoft Exchange and Active Directory. But Raven also keeps indexes of the data in Lucene which it uses when performing queries.

The errors above usually happen when there's a problem with those Lucene indexes. I've noticed that customers upgrade Octopus or restore a backup, only to find data missing. But if you navigate into the RavenDB management studio, you can often see the data. It's there, it's just not indexed.

Connecting to the RavenDB Management Studio in Octopus

A quick solution that tends to work is to reset the indexes, and then restart Octopus so that all the indexes are rebuilt. To do this, you first need to connect to the RavenDB management studio.

The studio is built in Silverlight, so you'll need to make sure sure that whatever machine you do this from has the Silverlight browser plugins installed (or you can install it).

The RavenDB Management Studio is usually available on port 10930 on your Octopus Server, so browsing to http://_your-octopus_:10930 usually works. If it doesn't, and you can get to the Octopus web portal, try:

1. Going to the Octopus web portal
2. Going to the Configuration menu, then the Storage tab
3. Following the link on the page to access Raven

If this doesn't work, you'll need to make sure firewall rules aren't preventing you from browsing to Octopus on that port. Or, you'll need to RDP to the Octopus server and browse to it locally (i.e., http://localhost:10930).

Finally, when you connect, you'll be asked to authenticate:

The credentials that you need to provide here are not your Octopus credentials. You need to authenticate as a Windows user that is assigned to the local Administrators group on the Octopus server (e.g., the Administrator account on the machine).

Resetting the indexes

Now that you are in Raven, you can go to the Indexes tab and click "Delete All Indexes":

Note that this isn't deleting any data; it's just deleting the indexes, which will be automatically rebuilt in the next step.

Restarting Octopus

To make Octopus rebuild the indexes, you'll need to restart it.

1. RDP to your Octopus server
2. Open the Services management console, services.msc
3. Select the "Octopus" service
4. Click Stop, and wait for it to stop
5. Click Start, and wait for it to start

You'll also need to restart the Octopus web portal, which is an IIS application.

1. Open the IIS Manager, inetmgr.exe
2. In the tree, expand your server, and go to Application Pools
3. Select the Octopus Portal application pool
4. Perform a Stop and a Start

Give it a minute or so to warm up and then go to the Octopus web portal via your browser, and hopefully everything should be working. If you're still having trouble, get in touch on our support form.