Migrating content from old forums

Hello everyone,

I’m aware that a some people have raised this question in different channels and also through some consultations done in the “technical beta” we run last year.

Let’s have a space here to go through the current situation and allow everyone to share their thinking. We understand that there is not a decision that will make everyone happy, but at least we want to make sure that most people is happy and also be fair with the technical and people/time constrains to execute any solution.

I would suggest we get out of the scope of this conversation the Help OSM site, which user data base is not connected with OSM accounts and the software doesn’t have an easy way to extract the data into a discourse forum.

Let’s talk about forum.openstreemap.org since it’s the other site we know we need to retire, as explained in the next steps post.

1. Agreeing on what are we trying to solve

These are some of the asks/needs people have been sharing (please add missing ones on the comments below):

  • We would like be able to reference old topics in the future.
  • We would like the existing urls to point to the old topics, so we don’t break many places where they are linked.
  • We would like to be able to search through old content.
  • We would like to talk again about discussions that happened in the past.
  • We have invested a lot of time producing content and we don’t want that to disappear.

2. Understanding what are our options and their implications

Note there is no option where old data is not preserved.

Option 1: We migrate all content from the old forums

* Note that we don’t know for sure if the existing 2016 script for migration is still working with the most recent Discourse versions.

Pros:

  • All content is searchable from Discourse.
  • Old content might (or might not) be linked to people’s accounts.
  • People will be able to continue the conversation of old topic on the topic itself.

Cons:

  • Some users might not match with Discourse users or we’ll encounter conflicts due different DB restrictions. Last tests we run encountered too many issues linking users and orphaned topics.
  • Content encoding might end up different.
  • URLs will be broken, since we don’t have a system to match old urls with the topics imported here.
  • We’ll tremendously increase the discourse DB and maintenance in this early phase. We can’t anticipate other unknown risks.
  • Search engines will have to re-index content and we might get a penalty for duplicated content.
  • Time investment can vary from high to high-unknown: We’ll require from an Ops person to handle the whole process. AFAIK, currently we don’t have anyone with this kind of bandwidth.
  • There is no guarantee all content and users will end up looking correctly after the migration.

Option 2: We archive the old forums into a static site

Pros:

  • URLs can easily be matched for redirects to the archive: Old links will keep working.
  • The process is way lighter in Ops time.
  • The content is preserved as it was posted.
  • The content is searchable and accessible from search engines.
  • There are no compromises to Discourse DB or its maintenance.

Cons:

  • New discussions here about old topics need to link the archived conversation for reference.
  • Users here don’t have their old posts linked to their account.
  • Search will need to be done through a search engine.

3. Questions for everyone

  • What are we missing from the things we are trying to solve? If something missing, why it’s important for you?
  • What’s missing from the pros and cons on the options listed?
  • Are there other alternative options you can think of? What are their pros and cons?

We’ll keep this first post as a living document and it will be updated it with the discussion happening below.

This proposal will be open for input and feedback until March 29th.

After that date the @forums-governance team will take in consideration all the input and take a decision.

Please note that we all understand how important is to preserve historical content, discussions and decisions. Let’s keep a constructive tone and open mind here, understanding how everyone feels and what resources we have at our disposal to solve this issue.

Thanks!

79 posts - 20 participants

Read full topic


Ce sujet de discussion accompagne la publication sur https://community.openstreetmap.org/t/migrating-content-from-old-forums/446