","https://gist.github.com/mikelehen/3596a30bd69384624c11","https://gist.github.com/3596a30bd69384624c11.git","1","492","@amirh","https://avatars.githubusercontent.com/u/1024117","@gardun0","https://avatars.githubusercontent.com/u/3385668","@arturictus","https://avatars.githubusercontent.com/u/1930175","@adamkoncz","https://avatars.githubusercontent.com/u/3645708","@robdelacruz","https://avatars.githubusercontent.com/u/4341542","@AhmedOS","https://avatars.githubusercontent.com/u/21817835","@franklinharvey","https://avatars.githubusercontent.com/u/19939042","@shen-tian","https://avatars.githubusercontent.com/u/4621961","@bhupiister","https://avatars.githubusercontent.com/u/58903577","@ghacosta","https://avatars.githubusercontent.com/u/10713501","@sinuke","https://avatars.githubusercontent.com/u/25010867","@innomon","https://avatars.githubusercontent.com/u/3252607","@JuliusJacobitz","https://avatars.githubusercontent.com/u/47418007","@alcarazolabs","https://avatars.githubusercontent.com/u/7144988","@douwevdijk","https://avatars.githubusercontent.com/u/4209496","@marcopeg","https://avatars.githubusercontent.com/u/427569","@chilts","https://avatars.githubusercontent.com/u/3048","@iboxgithub","https://avatars.githubusercontent.com/u/5391993","image","https://user-images.githubusercontent.com/5391993/225704470-b4715d8e-acbf-41a7-a46b-0d5cecb13767.png","https://user-images.githubusercontent.com/5391993/225724712-60b85d30-0fc8-4d77-80bd-968f0ff58455.png","@swftvsn","https://avatars.githubusercontent.com/u/25097010","@dbechrd","https://avatars.githubusercontent.com/u/707367","@Muhammadamjadm","https://avatars.githubusercontent.com/u/112027108","JavaScript code for generating Firebase Push IDs · GitHub","Skip to content","\n All gists\n\n Back to GitHub\n\n \n Sign in\n\n \n Sign up\n","\n \n Sign in\n \n Sign up\n ","You signed in with another tab or window. Reload to refresh your session.","You signed out in another tab or window. Reload to refresh your session.","You switched accounts on another tab or window. Reload to refresh your session.","Dismiss alert","{{ message }}","\n Instantly share code, notes, and snippets.\n ","\n mikelehen/generate-pushid.js\n ","\n Created\n ","February 11, 2015 17:34","Show Gist options","\n Download ZIP\n","Star","\n (493)\n 493\n ","You must be signed in to star a gist","Fork","\n (101)\n 101\n ","You must be signed in to fork a gist","Embed","\n \n Embed\n Embed this gist in your website.\n","\n \n Share\n Copy sharable link for this gist.\n","\n \n Clone via HTTPS\n Clone using the web URL.\n","\n Learn more about clone URLs\n\n","\n Clone this repository at <script src="https://gist.github.com/mikelehen/3596a30bd69384624c11.js"></script>\n","Save mikelehen/3596a30bd69384624c11 to your computer and use it in GitHub Desktop.","\n Code\n","\n Revisions\n ","\n Stars\n ","\n Forks\n ","Download ZIP","\n JavaScript code for generating Firebase Push IDs\n ","Raw","\n generate-pushid.js\n ","\n This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.\n Learn more about bidirectional Unicode characters\n "," Show hidden characters\n","/**"," * Fancy ID generator that creates 20-character string identifiers with the following properties:"," *"," * 1. They're based on timestamp so that they sort *after* any existing ids."," * 2. They contain 72-bits of random data after the timestamp so that IDs won't collide with other clients' IDs."," * 3. They sort *lexicographically* (so the timestamp is converted to characters that will sort properly)."," * 4. They're monotonically increasing. Even if you generate more than one in the same timestamp, the"," * latter ones will sort after the former ones. We do this by using the previous random bits"," * but \"incrementing\" them by 1 (only in the case of a timestamp collision)."," */","generatePushID = (function() {"," // Modeled after base64 web-safe chars, but ordered by ASCII."," var PUSH_CHARS = '-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz';"," // Timestamp of last push, used to prevent local collisions if you push twice in one ms."," var lastPushTime = 0;"," // We generate 72-bits of randomness which get turned into 12 characters and appended to the"," // timestamp to prevent collisions with other clients. We store the last characters we"," // generated because in the event of a collision, we'll use those same characters except"," // \"incremented\" by one."," var lastRandChars = [];"," return function() {"," var now = new Date().getTime();"," var duplicateTime = (now === lastPushTime);"," lastPushTime = now;"," var timeStampChars = new Array(8);"," for (var i = 7; i >= 0; i--) {"," timeStampChars[i] = PUSH_CHARS.charAt(now % 64);"," // NOTE: Can't use << here because javascript will convert to int and lose the upper bits."," now = Math.floor(now / 64);"," }"," if (now !== 0) throw new Error('We should have converted the entire timestamp.');"," var id = timeStampChars.join('');"," if (!duplicateTime) {"," for (i = 0; i < 12; i++) {"," lastRandChars[i] = Math.floor(Math.random() * 64);"," }"," } else {"," // If the timestamp hasn't changed since last push, use the same random number, except incremented by 1."," for (i = 11; i >= 0 && lastRandChars[i] === 63; i--) {"," lastRandChars[i] = 0;"," lastRandChars[i]++;"," for (i = 0; i < 12; i++) {"," id += PUSH_CHARS.charAt(lastRandChars[i]);"," if(id.length != 20) throw new Error('Length should be 20.');"," return id;"," };","})();","Load earlier comments...","\n \n Copy link\n\n","amirh","\n\n \n\n commented \n\n\n ","Mar 14, 2018","Dart port: https://gist.github.com/amirh/0e7b480691f7e6506317a7600398124b","\n Sorry, something went wrong.\n ","gardun0","May 3, 2018","Im so grateful finding this ❤️ thank you","arturictus","May 13, 2018","port in elixir https://hex.pm/packages/firebase_pushid","adamkoncz","Jun 13, 2018","Did pushid changed in firebase lately? None of the thousands of ids generated contains dash (-) or underscore (_). Or anything apart from alphanumeric characters.","robdelacruz","Jun 28, 2018","Perl port: https://gist.github.com/robdelacruz/25d1bc5a7ea72bd86088d2f28f2ba5f9","AhmedOS","Oct 8, 2018","Swift 4: https://gist.github.com/AhmedOS/512531b74da37f34331ecb206c77c20a
\nModified version of pgherveou's one.","franklinharvey","Mar 7, 2019","•","\n edited\n \n ","Loading","Made a JSFiddle: https://jsfiddle.net/feLys02r/1/","It's the original script with a button to copy to clipboard.","shen-tian","Nov 20, 2019","ClojureScript: https://gist.github.com/shen-tian/3b6ff888e604abb4bf5bdead07062744","bhupiister","Apr 16, 2020","Port in Unity C#: https://gist.github.com/bhupiister/05dbb860e9064c43e8176b591f11e555
\nJust paste this script into unity and assign text field and function on a button.
\nThank you to the original content maker","ghacosta","Mar 14, 2021","here is the codesandox for this code!
\nhttps://codesandbox.io/s/firebase-push-id-generator-080cb","sinuke","Apr 3, 2021","Delphi
\nhttps://gist.github.com/sinuke/1e8ffe14ffa9eb331397c9ee707fc83b","innomon","Jun 19, 2021","Thanks @mikelehen & @Flyclops , for sharing the code.
\nMy usecase is that we are transforming a firebase application (which uses firebase auth) and generates an uuid based on the above algorithm, however, we needed to convert it to a W3C DID, here is a simple code","var gb64 = \"-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz\"\nfunc genB32(m string) (string, error) {\n encb64 := base64.NewEncoding(gb64)\n dec, err := encb64.DecodeString(m)\n if err != nil {\n return \"\", err\n } \n return base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(dec), nil\n}","see line Line 77 of https://play.golang.org/p/vnCZTqOjZp9","JuliusJacobitz","Oct 11, 2021","Thank you for sharing <3","alcarazolabs","Nov 21, 2021","Best way:
\n`
\nfunction generatePushID() {"," // Modeled after base64 web-safe chars, but ordered by ASCII.\n var PUSH_CHARS = '-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz';\n\n // Timestamp of last push, used to prevent local collisions if you push twice in one ms.\n var lastPushTime = 0;\n\n // We generate 72-bits of randomness which get turned into 12 characters and appended to the\n // timestamp to prevent collisions with other clients. We store the last characters we\n // generated because in the event of a collision, we'll use those same characters except\n // \"incremented\" by one.\n var lastRandChars = [];\n\n\n var now = new Date().getTime();\n var duplicateTime = (now === lastPushTime);\n lastPushTime = now;\n\n var timeStampChars = new Array(8);\n for (var i = 7; i >= 0; i--) {\n timeStampChars[i] = PUSH_CHARS.charAt(now % 64);\n // NOTE: Can't use << here because javascript will convert to int and lose the upper bits.\n now = Math.floor(now / 64);\n }\n if (now !== 0) throw new Error('We should have converted the entire timestamp.');\n\n var id = timeStampChars.join('');\n\n if (!duplicateTime) {\n for (i = 0; i < 12; i++) {\n lastRandChars[i] = Math.floor(Math.random() * 64);\n }\n } else {\n // If the timestamp hasn't changed since last push, use the same random number, except incremented by 1.\n for (i = 11; i >= 0 && lastRandChars[i] === 63; i--) {\n lastRandChars[i] = 0;\n }\n lastRandChars[i]++;\n }\n for (i = 0; i < 12; i++) {\n id += PUSH_CHARS.charAt(lastRandChars[i]);\n }\n if(id.length != 20) throw new Error('Length should be 20.');\n\n return id;\n }\n \n console.log(generatePushID());\n","`","douwevdijk","Dec 22, 2021","Hi All,","Is there a possibility to have a pure numeric version?","marcopeg","Dec 1, 2022","Hello, here you can find a porting for PostgreSQL:
\nhttps://github.com/marcopeg/amazing-postgresql/tree/main/projects/firebase-pushid","chilts","Dec 13, 2022","Oh nice! Thanks for noting this here @marcopeg. :)","iboxgithub","Mar 16, 2023","when I use this all my IDs have a \"-\" as a prefix: can someone help me understand why?","it's normal. '-' is the first character of the alphabet.","what do you mean? having the same first character should reduce the amount of potential combinations no?","Author","mikelehen","The beginning of the push ID is based on the current timestamp. This is what makes push IDs sort chronologically. Around the year 2110 the first character will change from a ","-"," to a ","0",".","The beginning of the push ID is based on the current timestamp. This is what makes push IDs sort chronologically. Around the year 2110 the first character will change from a ","thanks that makes sense, now do we know why isn't Firebase applying the same methodology?","\njust to weigh the pros and cons as it seems a bit weird Google would not apply the most rational approach no?","See https://stackoverflow.com/a/52741265 and https://firebase.google.com/docs/firestore/best-practices. In particular:","Do not use monotonically increasing document IDs ... Such sequential IDs can lead to hotspots that impact latency.","Basically monotonically increasing IDs can have performance implications so Firestore opted to avoid this by default. If you need chronological ordering you need to add a separate timestamp field to your documents and sort by that.","ok thanks!","swftvsn","Mar 17, 2023","@iboxgithub This ID is for real time database, NOT firestore.","ha...is there a reason why? we use it in Firestore already :(","This key was designed before Google bought Firebase. It allows automatic sorting by time using only the key in a gigantic json file, which real time database kinda is. This was a design choice of Firebase engineers. At that time Firestore did not exist.","After Google bought Firebase, it became evident that Real Time database is too limited. Firestore was designed to be an evolution of Google's Datastore, which is completely different beast. To guarantee the (almost) endless scaling of the database, it needs evenly distributed keys, which this is not. Firestore needs truly random keys to work properly. This, to my knowledge, is because they use multiple machines behind the scenes and divide the data to different machines by the key. Now, if the key is aaaa1, aaaa2, aaaa3... all of the docs end up in single machine creating a hot spot that hit's the limits quite fast.","If you don't have much data, you won't notice it. (The data is in single machine anyway.)","thanks ;)","dbechrd","Aug 5, 2023","Line 48 is a buffer underflow when the random bytes are ","zzzzzzzzzzzz"," (either randomly, or due to the same-millisecond increment behavior). It will attempt to increment ","lastRandChars[-1]",". In JavaScript, this will create a property on the object with key ","-1"," and value ","NaN",", but in other languages, it could cause far more insidious behavior.","In C, I chose to do this (rather than throw an error), but in your JS code, throwing an error seems more consistent with how other parts of it work.","// Prevent buffer underrun on overflow (when incrementing \"zzzzzzzzzzzz\")\n// Warning: If this gets skipped, the id is no longer guaranteed to be unique,\n// but most likely will be for quite awhile (it depends on where the random\n// value for this millisecond started in the range).\nif (i >= 0) {\n lastRandChars[i]++;\n}","Muhammadamjadm","Feb 6, 2024","Itsok","\n \n Sign up for free\n to join this conversation on GitHub.\n Already have an account?\n Sign in to comment\n\n\n \n","Footer","\n © 2024 GitHub, Inc.\n ","Footer navigation","Terms","Privacy","Sicherheit","Status","Docs","Kontakt","\n Manage cookies\n ","\n Do not share my personal information\n ","\n You can’t perform that action at this time.\n "]}
Skip to content

Instantly share code, notes, and snippets.

@mikelehen
Created February 11, 2015 17:34
Show Gist options
  • Save mikelehen/3596a30bd69384624c11 to your computer and use it in GitHub Desktop.
Save mikelehen/3596a30bd69384624c11 to your computer and use it in GitHub Desktop.
JavaScript code for generating Firebase Push IDs
/**
* Fancy ID generator that creates 20-character string identifiers with the following properties:
*
* 1. They're based on timestamp so that they sort *after* any existing ids.
* 2. They contain 72-bits of random data after the timestamp so that IDs won't collide with other clients' IDs.
* 3. They sort *lexicographically* (so the timestamp is converted to characters that will sort properly).
* 4. They're monotonically increasing. Even if you generate more than one in the same timestamp, the
* latter ones will sort after the former ones. We do this by using the previous random bits
* but "incrementing" them by 1 (only in the case of a timestamp collision).
*/
generatePushID = (function() {
// Modeled after base64 web-safe chars, but ordered by ASCII.
var PUSH_CHARS = '-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz';
// Timestamp of last push, used to prevent local collisions if you push twice in one ms.
var lastPushTime = 0;
// We generate 72-bits of randomness which get turned into 12 characters and appended to the
// timestamp to prevent collisions with other clients. We store the last characters we
// generated because in the event of a collision, we'll use those same characters except
// "incremented" by one.
var lastRandChars = [];
return function() {
var now = new Date().getTime();
var duplicateTime = (now === lastPushTime);
lastPushTime = now;
var timeStampChars = new Array(8);
for (var i = 7; i >= 0; i--) {
timeStampChars[i] = PUSH_CHARS.charAt(now % 64);
// NOTE: Can't use << here because javascript will convert to int and lose the upper bits.
now = Math.floor(now / 64);
}
if (now !== 0) throw new Error('We should have converted the entire timestamp.');
var id = timeStampChars.join('');
if (!duplicateTime) {
for (i = 0; i < 12; i++) {
lastRandChars[i] = Math.floor(Math.random() * 64);
}
} else {
// If the timestamp hasn't changed since last push, use the same random number, except incremented by 1.
for (i = 11; i >= 0 && lastRandChars[i] === 63; i--) {
lastRandChars[i] = 0;
}
lastRandChars[i]++;
}
for (i = 0; i < 12; i++) {
id += PUSH_CHARS.charAt(lastRandChars[i]);
}
if(id.length != 20) throw new Error('Length should be 20.');
return id;
};
})();
@amirh
Copy link

amirh commented Mar 14, 2018

@gardun0
Copy link

gardun0 commented May 3, 2018

Im so grateful finding this ❤️ thank you

@arturictus
Copy link

@adamkoncz
Copy link

Did pushid changed in firebase lately? None of the thousands of ids generated contains dash (-) or underscore (_). Or anything apart from alphanumeric characters.

@robdelacruz
Copy link

@AhmedOS
Copy link

AhmedOS commented Oct 8, 2018

Swift 4: https://gist.github.com/AhmedOS/512531b74da37f34331ecb206c77c20a
Modified version of pgherveou's one.

@franklinharvey
Copy link

franklinharvey commented Mar 7, 2019

Made a JSFiddle: https://jsfiddle.net/feLys02r/1/

It's the original script with a button to copy to clipboard.

@shen-tian
Copy link

@bhupiister
Copy link

Port in Unity C#: https://gist.github.com/bhupiister/05dbb860e9064c43e8176b591f11e555
Just paste this script into unity and assign text field and function on a button.
Thank you to the original content maker

@ghacosta
Copy link

here is the codesandox for this code!
https://codesandbox.io/s/firebase-push-id-generator-080cb

@sinuke
Copy link

sinuke commented Apr 3, 2021

@innomon
Copy link

innomon commented Jun 19, 2021

Thanks @mikelehen & @Flyclops , for sharing the code.
My usecase is that we are transforming a firebase application (which uses firebase auth) and generates an uuid based on the above algorithm, however, we needed to convert it to a W3C DID, here is a simple code

var gb64 = "-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz"
func genB32(m string) (string, error) {
  encb64 := base64.NewEncoding(gb64)
  dec, err := encb64.DecodeString(m)
  if err != nil {
    return "", err
  }  
  return base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(dec), nil
}

see line Line 77 of https://play.golang.org/p/vnCZTqOjZp9

@JuliusJacobitz
Copy link

Thank you for sharing <3

@alcarazolabs
Copy link

Best way:
`
function generatePushID() {

        // Modeled after base64 web-safe chars, but ordered by ASCII.
        var PUSH_CHARS = '-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz';

        // Timestamp of last push, used to prevent local collisions if you push twice in one ms.
        var lastPushTime = 0;

        // We generate 72-bits of randomness which get turned into 12 characters and appended to the
        // timestamp to prevent collisions with other clients.  We store the last characters we
        // generated because in the event of a collision, we'll use those same characters except
        // "incremented" by one.
        var lastRandChars = [];


            var now = new Date().getTime();
            var duplicateTime = (now === lastPushTime);
            lastPushTime = now;

            var timeStampChars = new Array(8);
            for (var i = 7; i >= 0; i--) {
            timeStampChars[i] = PUSH_CHARS.charAt(now % 64);
            // NOTE: Can't use << here because javascript will convert to int and lose the upper bits.
            now = Math.floor(now / 64);
            }
            if (now !== 0) throw new Error('We should have converted the entire timestamp.');

            var id = timeStampChars.join('');

            if (!duplicateTime) {
            for (i = 0; i < 12; i++) {
                lastRandChars[i] = Math.floor(Math.random() * 64);
            }
            } else {
            // If the timestamp hasn't changed since last push, use the same random number, except incremented by 1.
            for (i = 11; i >= 0 && lastRandChars[i] === 63; i--) {
                lastRandChars[i] = 0;
            }
            lastRandChars[i]++;
            }
            for (i = 0; i < 12; i++) {
            id += PUSH_CHARS.charAt(lastRandChars[i]);
            }
            if(id.length != 20) throw new Error('Length should be 20.');

            return id;
        }
        
        console.log(generatePushID());

`

@douwevdijk
Copy link

Hi All,

Is there a possibility to have a pure numeric version?

@marcopeg
Copy link

marcopeg commented Dec 1, 2022

@chilts
Copy link

chilts commented Dec 13, 2022

Oh nice! Thanks for noting this here @marcopeg. :)

@iboxgithub
Copy link

when I use this all my IDs have a "-" as a prefix: can someone help me understand why?
image

@shen-tian
Copy link

when I use this all my IDs have a "-" as a prefix: can someone help me understand why?
image

it's normal. '-' is the first character of the alphabet.

@iboxgithub
Copy link

when I use this all my IDs have a "-" as a prefix: can someone help me understand why?
image

it's normal. '-' is the first character of the alphabet.

what do you mean? having the same first character should reduce the amount of potential combinations no?

@mikelehen
Copy link
Author

The beginning of the push ID is based on the current timestamp. This is what makes push IDs sort chronologically. Around the year 2110 the first character will change from a - to a 0.

@iboxgithub
Copy link

The beginning of the push ID is based on the current timestamp. This is what makes push IDs sort chronologically. Around the year 2110 the first character will change from a - to a 0.

thanks that makes sense, now do we know why isn't Firebase applying the same methodology?
just to weigh the pros and cons as it seems a bit weird Google would not apply the most rational approach no?
image

@mikelehen
Copy link
Author

See https://stackoverflow.com/a/52741265 and https://firebase.google.com/docs/firestore/best-practices. In particular:

Do not use monotonically increasing document IDs ... Such sequential IDs can lead to hotspots that impact latency.

Basically monotonically increasing IDs can have performance implications so Firestore opted to avoid this by default. If you need chronological ordering you need to add a separate timestamp field to your documents and sort by that.

@iboxgithub
Copy link

ok thanks!

@swftvsn
Copy link

swftvsn commented Mar 17, 2023

@iboxgithub This ID is for real time database, NOT firestore.

@iboxgithub
Copy link

@iboxgithub This ID is for real time database, NOT firestore.

ha...is there a reason why? we use it in Firestore already :(

@swftvsn
Copy link

swftvsn commented Mar 17, 2023

This key was designed before Google bought Firebase. It allows automatic sorting by time using only the key in a gigantic json file, which real time database kinda is. This was a design choice of Firebase engineers. At that time Firestore did not exist.

After Google bought Firebase, it became evident that Real Time database is too limited. Firestore was designed to be an evolution of Google's Datastore, which is completely different beast. To guarantee the (almost) endless scaling of the database, it needs evenly distributed keys, which this is not. Firestore needs truly random keys to work properly. This, to my knowledge, is because they use multiple machines behind the scenes and divide the data to different machines by the key. Now, if the key is aaaa1, aaaa2, aaaa3... all of the docs end up in single machine creating a hot spot that hit's the limits quite fast.

If you don't have much data, you won't notice it. (The data is in single machine anyway.)

@iboxgithub
Copy link

thanks ;)

@dbechrd
Copy link

dbechrd commented Aug 5, 2023

Line 48 is a buffer underflow when the random bytes are zzzzzzzzzzzz (either randomly, or due to the same-millisecond increment behavior). It will attempt to increment lastRandChars[-1]. In JavaScript, this will create a property on the object with key -1 and value NaN, but in other languages, it could cause far more insidious behavior.

In C, I chose to do this (rather than throw an error), but in your JS code, throwing an error seems more consistent with how other parts of it work.

// Prevent buffer underrun on overflow (when incrementing "zzzzzzzzzzzz")
// Warning: If this gets skipped, the id is no longer guaranteed to be unique,
// but most likely will be for quite awhile (it depends on where the random
// value for this millisecond started in the range).
if (i >= 0) {
    lastRandChars[i]++;
}

@Muhammadamjadm
Copy link

Itsok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment