Recently, I wanted to migrate my Pixelfed content over to a GoToSocial instance. Enter Slurp, a new CLI tool that’s making this process much easier. Here’s my experience, including some recent updates that streamline the process.
First, a big thanks to the creators of Slurp, Posty and GoToSocial for making this possible. It’s a game-changer for anyone looking to move their content between platforms.
The Struggles with Post Imports
Most fediverse platforms provide excellent data export packages - you can bring your followers with you, import your follows, lists, and blocked accounts to your new instance. But historically, posts have been left behind. Worse, people can still interact with your old posts and you’d never know. If you delete your account or the instance goes away completely, your posts are gone forever.
There have been other attempts at post imports but the complexity keeps most people away. Posts are created with current dates rather than being backdated. In some cases, the posts would appear as new, spamming your followers with notifications. This could overwhelm not only your instance but others as well. Other solutions required updating the core mastodon codebase, which is a non-starter for most people.
While platforms like Firefish/IceShrimp and Sharkey have had post importers, many instances have disabled them due to performance issues. Pixelfed has an Instagram importer, but results have been mixed and it isn’t enabled on all instances.
GoToSocial and Slurp manage this elegantly by allowing you to import posts with original dates that DO NOT federate on creation. Your posts get added to your timeline with original dates intact, only impacting your own server as it processes the imports.
How Slurp Works
Slurp is a CLI tool that interacts with the GoToSocial API to import posts from other platforms. It works with both Mastodon formatted archives and newer versions of the Pixelfed archive file. It requires a newer version of GoToSocial that includes a patch for importing backdated statuses. It’s been in some release candidates recently and is also in the latest beta release, v0.18.0.
Some general guidelines to understand up front:
- Backfilled statuses are not inserted into home or list timelines and do not generate notifications
- Only public and unlisted statuses are imported
- They aren’t federated on creation, but they can be federated later with boosts, likes, replies, etc.
- If a status mentions someone other than the author, the status will not be imported.
- If a status contains a custom emoji that doesn’t exist on the target instance, the status will not be imported.
- Is a status contains a poll, the status will not be imported.
- Replies, likes, boosts of the statuses are not imported.
Another item I wanted to note. Posts are going to be imported as PUBLIC or UNLISTED, just as you had them in Pixelfed. GoToSocial users may have different defaults set for their own posts (followers only for example). Be aware that your posts WILL show up on your public timeline on your GoToSocial instance.
My Setup
My GoToSocial instance runs from a homelab using a Portainer docker container. NGINX Proxy Manager directs traffic, and media attachments are stored in an S3 bucket rather than locally. Because Slurp uses the GoToSocial API, all of these complexities are handled automatically. It just works. If your setup is different, like you store images locally, it should still work the same.
Getting Started with Pixelfed Export
Let’s walk through getting your data out of Pixelfed. There’s currently no official help documentation for this process that I can find, so I’ll guide you through it.
First, navigate to Settings > Export in your Pixelfed instance. For example, on pixelfed.social it’s at https://pixelfed.social/settings/home. Another way to get to it is by clicking on your photo or the “Profile” link. But not the gear icon on your profile image…
It’s not as easy to find as you might expect, so here’s a screenshot to help:
You’ll be asked to enter “Sudo Mode” - this just means confirming your password to verify it’s you.
Once authenticated, you’ll see the Data Export menu. While you’re here, I strongly recommend downloading each available item for safekeeping, even if you don’t plan to use them immediately.
At the very least, you need the statuses. This contains all your posts, including text, links to media, and metadata. Click the “Download” button to start the process.
TIP: Name your backup archives with a date, like
ARCHIVE-userName-pixelfed-social-YYYYMMDD.zip. This is a good practice to maintain even if you’re not planning to move instances.
A few important notes about the export:
- Downloads start immediately after requesting them
- They’re surprisingly small because they only contain links to your media, not the media itself
- Slurp will handle downloading the actual media during import if necessary
Using Slurp
Slurp has recently added support for direct imports from Pixelfed exports. If you have an older export format, you can still use Posty to convert it to Mastodon format first.
Installation
Slurp is a Go module, so you’ll need to have Go installed on your system first.
1
2
3
4
git clone https://github.com/VyrCossont/slurp.git
cd slurp
go mod download
go build .
Create a working directory for your exported post data - this is also where Slurp will store its mapping files and downloaded media if necessary.
GoToSocial Configuration Updates
To reduce the risk of failure during the imports, you may want to bump up your post character and media attachment limits first in your configuration. This requires admin level rights on your GoToSocial instance.
This is especially important if you have a lot of media in your archive - like Pixelfed exports for example!
These updates are not required, and you could attempt the import without changing them. If failures occur, you can just run the process again after making these changes.
I would highly recommend upping the status-media-max-files limit as Pixelfed posts often have more than 4 images.
After making the changes, restart your GoToSocial instance to apply them.
These are found in your GoToSocial config file. I stupidly have them all as ENV variables in my Portainer setup, so I had to go in and change them there.
When using them as ENV variables, they look like this, with the GTS_ prefix:
1
2
3
4
5
# As seen in config file
statuses-media-max-files:6
# As seen as an ENV variable in Portainer
GTS_STATUSES_MEDIA_MAX_FILES=6
Here’s what the related items look like in the config file. Update accordingly for your needs.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Examples: [2097152, 10485760, 40MB, 40MiB]
# Default: 40MiB (41943040 bytes)
media-local-max-size: 40MiB
# Default emoji size limit (50KiB)
media-emoji-local-max-size: 50KiB
# Int. Maximum amount of media files that can be attached to a new status.
# Note that going way higher than the default might break federation.
# Examples: [4, 6, 10]
# Default: 6
statuses-media-max-files: 6
# Examples: [140, 500, 5000]
# Default: 5000
statuses-max-chars: 5000
Emoji Migration (Optional but Recommended)
Before importing your archive, you’ll want to authenticate to your old server and copy any custom emojis you used. Posts containing custom emojis that aren’t present in your new instance will be skipped during import. Here’s how you do it.
This will import ALL the emoji that existed on your old instance. If there are thousands… well, you might want to consider which ones you actually used and only import those.
You can skip this step if you don’t use custom emojis or don’t care about them. Slurp now has a flag that I’ll include in the import command to replace emojis with text if they can’t be found on the new instance.
Note: Replace the example accounts below with your actual account details.
First, authenticate to your old instance:
1
2
3
4
5
# Authenticate to old instance
./slurp --user [email protected] auth login
# Export emojis from the pixelfed server
./slurp --user [email protected] emojis export --inline --file emoji.json
The --inline option saves the emoji picture data along with metadata, which is useful if the old instance later disappears.
Then authenticate to GoToSocial as an admin user to import them.
If your new GtS account isn’t an admin, and you don’t have access to an admin account, you’ll need to ask your instance admin to import the emojis for you.
1
2
3
4
5
# Authenticate to GoToSocial as admin
./slurp --user [email protected] auth login
# Import emojis (requires admin rights)
./slurp --user [email protected] emojis import --file emoji.json
Importing Your Archive
Depending on the the version of Pixelfed you’re using, your statuses export may be named pixelfed-statuses.json or outbox.json. The outbox.json file is the older format, and you’ll need to use Posty to convert it to Mastodon format first. I’ll cover both methods below.
In both cases, I’ve added the --allow-missing-custom-emojis flag to the import command. This will replace any missing custom emojis with :emoji-name:. If you’ve imported ALL your utilized custom emojis, you can remove this flag.
If you have the newer Pixelfed export format (named pixelfed-statuses.json), you can import directly:
1
2
3
4
5
6
7
./slurp --user [email protected] archive import \
--format pixelfed \
--file pixelfed_export/pixelfed-statuses.json \
--attachment-map-file pixelfed_export/attachment-map.json \
--status-map-file pixelfed_export/status-map.json \
--attachment-directory pixelfed_export/media_attachments \
--allow-missing-custom-emojis
If you have an outbox.json, you’ll need to use Posty](https://posty.1sland.social/pixelfed) to create a Mastodon export, then use this command instead:
1
2
3
4
5
6
./slurp --user [email protected] archive import \
--format mastodon \
--file pixelfed_mastodon_export/backup \
--status-map-file pixelfed_mastodon_export/backup/status-map.json \
--attachment-map-file pixelfed_mastodon_export/backup/attachment-map.json \
--allow-missing-custom-emojis
The mapping files track progress and prevent duplicates if the import is interrupted. You can safely stop and restart the process - Slurp will skip previously imported content.
TIP: After starting the import, let it run. The process can take a while depending on how much media you have. Go have a cup of tea and check back later.
And that’s it! After Slurp finishes, you can check your GoToSocial timeline to see your posts. They’ll be backdated to their original timestamps and won’t spam your followers with notifications. It’s a great way to bring your content with you when you move instances.
Troubleshooting
Common Import Issues
A few types of posts consistently failed to import for me:
- Reblogs without content: These fail with “post has no content” errors. Seemed to only occur with the Posty version of the archive, but I assume it happens with the Pixelfed version as well, just not reported the same.
- Empty Pixelfed entries: Some posts appear completely empty in the Posty archive and errored out.
- Dead account mentions: Posts mentioning unknown usernames or deactivated fediverse accounts get flagged as invalid mentions. For me, I had a few posts that mentioned the street art creator’s instagram account names with
@instagramNamethat failed to import.
Here’s a typical error you might see for the most common one - no text, media or poll:
1
2
3
ERROR Couldn't post converted status status=752909631418482442
err="[POST /api/v1/statuses][400] statusCreateBadRequest"
errors="Error #01: status contains no text, media or poll"
Was it worth it?
Slurp worked surprisingly well! While not every post made it over, the ones that did maintained their original timestamps, text, hashtags and media attachments perfectly. Being able to import directly from Pixelfed is a great improvement.
As a final triumph here, I found that Impressia, a Pixelfed iOS app, was able to fetch my posts from GoToSocial without issue. This is a huge win for me, as I can now use the app to post all my street art finds. It has a beautiful interface for browsing photos and is a joy to use. Note that there are some quirks with the timelines, but I’m hoping they can be ironed out in future updates. Posting and interacting works well, and viewing my own posts is a breeze.
I’m thrilled that after almost 3 years in the fediverse, we finally have a tool that handles post migration exactly as needed - preserving timestamps without overwhelming federation. Looking forward to seeing how this evolves!
References
- Slurp CLI Tool
- Posty for Pixelfed
- GoToSocial Documentation
- GoToSocial Configuration File Example
- Dumbo Migration Tool
- Twitter Archive to Mastodon
- Mastodon Content Mover
- Example of Import Notification Issues
- GoToSocial Backdated Status PR
- GoToSocial v0.18.0 Release
- Pixelfed Help Documentation
- Go Installation Guide
- Impressia for Pixelfed iOS app
Comments
Total Interactions: 0Likes
No webmentions were found.
Posts, Re-Posts and Bookmarks
No webmentions were found.
Replies
No webmentions were found.