SkyGraph
@skygraph.art
Layered community detection, built on old gaming PC for fun, art and feeds by @witch.monster.
https://skygraph.art
Visualization of communities based on @jaz.bsky.social's Atlas
โซ->๐ข->๐ ->๐ฃ->๐ต
Opt out: @optout.skygraph.art
Clarification: With the next update SkyGraph will be handling blocks on the early stages of dataset import to exclude blocked users from moots and avoid interactions with them affecting the graph and community calculation.
Blocks data won't be included anywhere in the API, visualization etc.
Final version of the decay ladder:
- last 30 days likes and replies get 10x points
- last 60 days likes and replies get 5x points
- last 90 days likes and replies get 3x points
- last 360 days likes and replies get 1x points
with 360 days cut-off
Before all interactions were counted evenly no matter how old with logarithm being the only equalizer.
This could play a bad trick with older accounts: they had so many interactions that the algo couldn't "decide" which community to put them, especially at lower resolutions.
SkyGraph is getting algo upgrade that is going to weigh more recent interactions more:
- last 30 days likes and replies getting 10x points
- last 90 days likes and replies getting 5x points
- everything older than 90 days gets 1x points
It will help produce more balanced communities for everyone.
MWAHHHAHAHAHA! ๐
No more social gravity tyranny of the Bsky Elders.
Preparing a new mid-May Skygraph update.
Thus reminding you about the Privacy Policy.
Opt out here to exclude your data from the new update:
@optout.skygraph.art
You will be excluded from the dataset, API, search, leaders and detailed previews in the next update.
Data that skygraph.art collects;
- did+handle
- follows, like counts, reply counts
You don't need more for quality personalized feeds like "Home+" or โจ"For you"!
We're currently working on some prototypes with @redsolver.dev from @skyfeed.app based on skygraph.art community detection.
- Want more cozy experience? Try "My Nebula" or "My Constellation".
- None of the above worked? Try "My Gigacluster". This might be the case for newer or more isolated communities. e.g. "The Sims"-cluster.
Extra-simplified explanation on how to choose a feed:
- Start with "My Cluster".
- Too crowded?
Go level down to "My Galaxy".
- Too quiet?
Go level up to "My Supercluster".
"My Cluster". Clusters are usually well balanced, we recommend this feed to most users (including English and Japanese) as they provide good and even distribution of communities.
If the feed still looks too crowded or unfamiliar, we have 3 more feeds ๐
"My Constellation".
Constellations are the smallest, they are at the end of the spectrum ๐, so it can be hit or miss.
For users with a lot of interactions overall, they are more likely to be a miss.
"My Nebula". Nebulas can get up to 6k users, so they might still be a candidate for "Home" feed for some users.
And we have one more feed to go, the smallest one "My Constellation" ๐
"My Galaxy". This will be a sweet spot for many users of not too crowded language metaclusters and for some niche English and Japanese sub-communities like artists, furries etc.
And we have 2 more feeds to go for more cozy experience (close social circles) ๐
"My Cluster". Clusters are usually well balanced, we recommend this feed to most users (including English and Japanese) as they provide good and even distribution of communities.
If the feed still looks too crowded or unfamiliar, we have 3 more feeds ๐
"My Supercluster".
Superclusters can get huge, but they have better distribution than Gigaclusters, so will be useful for a lot more users.
Check it out and if the feed looks too crowded or unfamiliar, let's go 1 level down ๐
How to choose the right feed out of 6?
"My Gigacluster" is the largest. Since it's at the end of the spectrum ๐ it likely won't work for a lot of users of English or Japanese language.
If it worked for you - great! If not - no worries, we have 5 more ๐
๐งต
It's not an all-in-one solution like "For you" (yet). But it's a building block for one. "My Nebula/Galaxy" might work better for some users and "My Cluster" will work better for others.
We're open sourcing these building blocks so that more powerful "For you"-like feeds can be built by community
More feeds here:
bsky.app/profile/skyg...
Time for some cool feeds on bsky!
@redsolver.dev has built these feeds for you based on skygraph.art community detection data
A lot of people might find these useful as-is, but these feeds are just the first baby steps - building blocks, that can go into a more complicated Home/Discovery feed.
๐งต๐
Hi! Reaching out for feedback. I know many people are missing "for you" and I'm trying to provide open source solution for "for you"-like feeds. So it's not company-gated anymore.
I believe this one should be similar to "For you" experience. Check it out, hope you like it!
But despite that communities eventually DO form around topics as people with similar interests tend to interact with each other more, so all you need to do to "form" a community is - talk to each other. And the next skygraph dataset update will pick it up.
Please do explore skygraph.art visualization to get a better feel what communities are. This visual is based on Atlas by @jaz.bsky.social if you're familiar with that one.
Communities are based on mutual interactions only. Algorithm doesn't know anything about content of the posts or profiles.
just dropping this here as the thread started with user asking about "For you" feed
It's not a fit-for-all home feed you're used to, but it's definitely worth checking out ๐
it's all open source too, so it will exist as long as people like it and as long as there are willing devs to maintain it
Time for some cool feeds on bsky!
@redsolver.dev has built these feeds for you based on skygraph.art community detection data
A lot of people might find these useful as-is, but these feeds are just the first baby steps - building blocks, that can go into a more complicated Home/Discovery feed.
๐งต๐
If Constellation, Nebula or Galaxy didn't work for you, which may happen for crowded clusters - don't forget to try out "My Cluster" or even higher layer feeds.
At least one of them should hit the spot.
The assumption is it's based on community size but we need your feedback to confirm.
"My Cluster" - top layer for most of the medium-sized language clusters and wider groups of interests.
Some Clusters can get tens of thousands of accounts, so the ranking of this feed is a bit more agressive.
bsky.app/profile/did:...
๐
From what I've observed so far - "My Nebula" will be the sweet spot for most users.
But sometimes Nebulas can get a bit too small, then either "My Galaxy" or "My Cluster" will be the feeds of choice. In rare cases it may even be Super or Gigacluster.
Let me know which worked best for you!
"My Gigacluster" - these communities can get hundreds of thousands of accounts. Most agressive ranking.
Might be good for discovery or filling in the gaps in other feeds, but it's expected if most people will find them too large to be useful.
bsky.app/profile/did:...
"My Supercluster" - these communities can get really huge, hence - agressive ranking.
bsky.app/profile/did:...
๐
"My Cluster" - top layer for most of the medium-sized language clusters and wider groups of interests.
Some Clusters can get tens of thousands of accounts, so the ranking of this feed is a bit more agressive.
bsky.app/profile/did:...
๐
"My Galaxy" - one of the most stable communities. Large enough to provide a self-sustainable feed, while still feeling "like home".
Again, like with other community layers - results may vary for individual users.
bsky.app/profile/did:...
๐
"My Nebula" - widening the circle, for many it can be the "sweet spot" for the home feed.
bsky.app/profile/did:...
๐
"My Nebula" - widening the social circle, for many it can be the "sweet spot" for the home feed.
bsky.app/profile/did:...
"My Constellation" - this one should feel the most cozy as it's usually the smallest interaction circle around you.
This will vary from user to user.
bsky.app/profile/did:...
๐
These are experimental building blocks and we'd like to collect more feedback before we proceed with somehing more advanced that will work for everyone.
Now, let's explore the feeds!
๐
Results may vary for each user. The more mutual interactions you have - the better feeds will be.
All the feeds are built on March 27th 2024 dataset.
If you registered after that date or didn't have enough interactions at the time - feeds will be empty for you.
๐
Note that the feeds have about 40% coverage of all bsky accounts.
To get a better feeling on how each of the feeds work, visit
skygraph.art, find your communities, explore them on the global map.
This is still very experimental.
๐
Time for some cool feeds on bsky!
@redsolver.dev has built these feeds for you based on skygraph.art community detection data
A lot of people might find these useful as-is, but these feeds are just the first baby steps - building blocks, that can go into a more complicated Home/Discovery feed.
๐งต๐
Yes, quality community detection is essential.
Current state of the art: Leiden algorithm.
Only Neo4j v.5.17+ (last November update) rocks it as part of their GDS. My friend mentioned it to me when playing with my early work with Louvain and updating to a new Neo4j. Amazing improvement over Louvain
From Louvain to Leiden: guaranteeing well-connected communities - Scientific Reports
Scientific Reports - From Louvain to Leiden: guaranteeing well-connected communities
www.nature.com
Might actually do that right away. Here you go:
github.com/witchmonster...
GitHub - witchmonster/skygraph.art: Layered community detection
Layered community detection. Contribute to witchmonster/skygraph.art development by creating an account on GitHub.
github.comnot too much, as Bsky is small. fits into old gaming PC (for now). I think I can keep it up up to 20 mil users or so. then - it will need community effort and crowdfunding to continue. and I'd be probably bored with it by then, so I'll just open source my queries and hope y'all pick it up from there
it's not the feed code that is important, it's community detection stuff and the algorithms behind it + a lot of work for each re-calculation of communities. I don't know of the method where you could do it on the fly. and it takes a lot of resources too.
open-sourcing things matter
a company played with community detection, figured there's no money in it - and "For you" is gone
whenever I get bored re-calculating communities (the actual stuff you need for a good home feed) - someone else can just pick it up, crowd source the infra costs and boom
skygraph.art provides the following aggregated data as part of it's API for community developers to be able to make feeds, wordclouds and more:
your did <=> [community codes you're in]
Note that skygraph.art API licence explicitly prohibits commercial use.
github.com/witchmonster...
skygraph.art DOESN'T collect anything else besides listed in the post above, but especially not:
- profile/bio
- post content or language
- blocks
except for blocks of @optout.skygraph.art, which I use to determine opted out users
Data that skygraph.art collects;
- did+handle
- follows, like counts, reply counts
You don't need more for quality personalized feeds like "Home+" or โจ"For you"!
We're currently working on some prototypes with @redsolver.dev from @skyfeed.app based on skygraph.art community detection.
I've also updated the UI to remove nodes from the graph and search, that have opted out with @optout.skygraph.art.
Opted out users will be completely removed from the input dataset with the next data update.
Current dataset: April 2024.
Next dataset update: ~mid May 2024.
Both largest Japanese Gigaclusters look like Shai Hulud faces.
Left:
skygraph.art?layout=peaco...
On global map:
skygraph.art?hi=false&ml=...
Right:
skygraph.art?layout=peaco...
On global map:
skygraph.art?hi=false&ml=...
I've also updated the UI to remove nodes from the graph and search, that have opted out with @optout.skygraph.art.
Opted out users will be completely removed from the input dataset with the next data update.
Current dataset: April 2024.
Next dataset update: ~mid May 2024.
Current thresholds for previews:
- minimum of 100 total mutual interaction score: sum of all relationship weights to all moots
- minimum mutual interaction score of X to anyone on a current preview
default: X=20
over 50k nodes: X=50
over 100k nodes: X=100
You may now preview ๐ every ๐ค Gigacluster, ๐ Supercluster and ๐งก Cluster of size 100 or more
skygraph.art
Each preview has color overlay showing sub-communities:
- ๐ค Gigaclusters and ๐ Superclusters previews have ๐งก Clusters as color overlays
- ๐งก Clusters previews have ๐ Nebulas as color overlays
Choosing .art๐จ domain was not a random choice for our ๐.
Meet "The Peacock ๐๐ฆ" and "The Pawprints ๐พ๐ฃ" layouts.
The difference between the two is only visual.
๐๐ฆ has hidden gravity sources that organize community layers into "feathers"
๐พ๐ฃ shows (arguably) more realistic gravity.
you may view your moots and interactions through detailed preview
here's direct link just in case:
skygraph.art?layout=peaco...
happy skyversary, by the way ๐ฅณ
there's a preview available for the cluster you're in: skygraph.art?hi=false&ml=...
just click "Preview community"
