Maybe someone smarter than me can explain things, but It’s been about a month since I’ve started the process of creating a magazine to support the reddit/discord community I’ve helped mod for the past 4 years… but I’ve noticed that zero posts show up in google search.
Lot’s of communities are indexed, some tags are, but not really any posts (unless I don’t know how to search, I assumed site://kbin.social was enough to skim for this). Compared to something like: site://lemmy.world where individual posts are indexed.
Is it just because it’s new(er)? Is there something technically wrong with Kbin that it’s taking so long to be crawled? (I thought maybe some noindex was setup but that doesn’t seem it)
Until posts start getting indexed at a relatively decent rate, even slowly, I can’t antipate any progress in adoption since that typically is a large driving factor, at least for my niche. I know that for the subreddit part that was always a large source of traffic.
While I’m being social - can someone explain what Tags and Badges are?
Thanks.
On one hand, I support a strong robots.txt being in place. It keeps data from being used by honest engines (though what constitutes “honesty” varies). But at the same time, indexing and caching is how we can grow. If you want the site to grow, you want to get it to 1st position on first SERP.
It’s a tricky balance.
kbin.social disallows crawlers to index the site. And honest crawlers will honor that.
https://kbin.social/robots.txtThis actually seems to be the standard configuration that kbin ships with, so most instances will have that in place.
This seems like a roadblock to people discovering any community here. It removes the Reddit effect, where adding ‘Reddit’ to a search led to non-ad posts with first-person relevance, and eliminates the easiest way for someone who’s not part of the federated universe to find the federated universe.
Counterproductive, seems to me.
@raphael Why is that? To prevent the site and posts becoming ad focused? I mean I can see that as a benefit. But on the other side, content that is not indexed by search machines is not much different from Discord. I always criticized Discord content for not being indexed, when arguing with my bro, and tell why Reddit (or any other open platform) is much better to post. Honestly, kbin disallowing crawlers to index the site is a big blow to me! I don’t like that at all.
Thanks, I totally forgot that robots.txt existed. Since you are replying, do you know the difference between badges and tags?
I wonder if there some way for instances to use the canonical tag to point to the original and make it less annoying for search engines.
That said, I guess id rather see it crawled and let search engines figure it out then have it not crawled at all. Not really sure where that feedback goes though.
As far as I am aware, Badges are intended to be similar to reddit’s post flair system? They don’t seem to do anything at the moment though.
Community moderators can set up badges but even with those in place you can’t see them when making a post.
