I'm a little confused about the issue description, because "mov" is not a MIME type.
Examples of MIME types: "text/plain", "text/html", "image/png" "application/pdf", "video/quicktime", ...
If I was prevented from using the username "wcoenentext/html", then I wouldn't really be bothered by that. (Although I might question the design decisions that would necessitate such a restriction.)
I’m the author of the issue on Gitlab (small world, isn’t it ?)
Yes the message is confusing and I agree that .mov isn’t a MIME type but I was merely reporting the error message shown ( plus, they added .mov in their list of file types and had aliased it to .mp4 format, please see: https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/in... )
> they added .mov in their list of file types and had aliased it to .mp4 format
That’s weird. Why’d they do that. They should make a separate entry for mov and associate it with video/quicktime
Guess it might be something related to https://stackoverflow.com/a/44785870 but like they point out, mov is a container format that can contain one of many different codecs used. And isn’t mp4 just a container too? Referring to mov files as video/mp4 seems straight up incorrect to me
I don’t think that’s quite right is it? As I understand, mp4 is based on mov. But they still very much are distinct container formats, and an implementation of the mp4 standard would likely not be able to correctly read mov files as is, would it?
They're very close. The ISO base media file format was directly based off of QuickTime container format.
If you look at a .mov file and a .mp4 file in a ISO bmff viewer, you'll generally see the only difference is the ftyp box is different ("qt " for .mov, "isom" for .mp4). Indeed, if you ask ffmpeg to make a .mov file and .mp4 file of the same content, literally the only difference is the contents of the "ftyp" box, every other byte is identical.
> literally the only difference is the contents of the "ftyp" box, every other byte is identical.
Not quite. There are some boxes acceptable in one but not the other. Strings inside MOOV are length-prefixed in MOV but null-terminated in ISOBMFF. There are a variety of differences like that.
The set of codecs allowed, also differs between the two.
They're pretty much interchangable. Video frameworks like gstreamer just give you qtdemux to parse .mov, .mp4, and even .m4a or .m4v (which are just MP4s with different file extensions).
Musk wasn’t the first to try to use weird names, but he didn’t quite succeed with that one. Quoth wiki:
"""the name would have violated California regulations as it contained characters that are not in the modern English alphabet,[321][322] and was then changed to "X Æ A-Xii". This drew more confusion, as Æ is not a letter in the modern English alphabet.[323] The child was eventually named "X AE A-XII", with "X" as a first name and "AE A-XII" as a middle name.[324]"""
> with "X" as a first name and "AE A-XII" as a middle name
It sounds as though they allowed the middle name to contain a space, which doesn't match my mental model but perhaps that's how they do it in California. It invites the question: do they allow a first name to contain a space, so that ("X", "AE A-XII", "Last Name") and ("X AE", "A-XII", "Last Name") are different names?
EDIT: The funny thing isn't allowing spaces in a middle name; it's have a separate field for middle name(s) at all. Modern passports have just two name fields: "surname" and "given names". Both may contain spaces. But according to images on the web, Californian birth certificates really do have three name fields.
Any name can contain a space. For example "Ana Maria" is a common first name which contains a space. On official documents, generally a name will be separated into a given name and surname. In this case "<A> <B C>" and "<A B> C" are considered separate names.
Source: I have a space in my name and some of my different identity documents have the name as "<A> <B C>" or "<A B> C", which causes all sorts of administrative problems.
Leonardo da Vinci is another famous example of a last name with two words in it. It's very common in romance languages. Plus, lots of people in the American south just flat out have two first names or two middle names.
Last names in many places evolved from that same need to disambiguate between people though. Attach some marker of connection to a place (common in Finland, e.g. Joensuu meaning "mouth of river"), profession (common in Germany and UK, e.g. Müller, Cooper, meaning mill worker and barrelmaker), lineage (common in Iceland, e.g. Grímsson meaning "son of Grímur"), or some other culturally relevant characteristic.
Nowadays the meanings of our last names have largely disappeared, so you have countless Coopers who have never touched a barrel in their lives, whose children will be called Cooper also, despite that. I think it's a little sad that so much of what people call us is semantically equivalent to a random UUID with tons of namespace collision. With that in mind, I'd say "da Vinci" is more a last name than most of us have.
Having a name with "of Region/city/former kingdom/..." is often their last name. Unless you want to claim that these people do not have a last name.
I can understand that in some cultures this might seem weird or antiquated but here in Germany these names are reality. Sometimes people with such names are descendents of royalty and sometimes someones last name "from family-name" is thier last name and happens to historically correspond to one of germany's state names or city names or just a little town.
One a side note: In Germany in 1919-1920 royalty was no longer a legal aspect that changed how laws applied to you[1]. When that happened titles that were reserved for ruling functions (king, grand duke) were removed and all other titles were moved to be part of the persons name (such as prince etc.) and could not be decreed on anyone new. These titles still exist in Germany but are simply naming "conventions" in a formerly royal family.
Correct: his full name as given was "Lionardo di ser Piero da Vinci," meaning more or less Piero's son Lionardo from Vinci. Deriving a childs name from their lineage was incredibly common.
Documents aside, do you consider <B> to be a given name (parent came up with it) or a surname (parent already had it)? If given, do you consider it optional (i.e., middle rather than first)? Sorry in advance for any shortsighted assumptions about the possibilities here!
A UK passport has a "surname" field and a "given names" field. But a recent UK birth certificate has a single name field in which the "surname" part is distinguished by being written in capitals, so "Peter James ADAM SMITH" would have a surname consisting of two words. But what if a word consists of a single letter? For example, some Irish surnames look like "O Briain", and I think there is a Vietnamese name that consists of a single vowel, so presumably you can't always tell from the birth certificate which part is the surname.
In Italy there is First Name, Other Names, and Family Name on the birth certificate. All fields can contain spaces (as well as hyphens, apostrophes, and some other diacritics).
The legal name however consists of First and Family names only, without the other names. Therefore many people have two names in the First Name field, usually separated by a space. The disadvantage is that in all official forms they have to spell out all the first name(s), no omissions or abbreviations are generally admitted.
One nice side-effect of this is that you can have a bit of fun with the Other Names since they're "unofficial". If the child wants to use those names later, they can, if they don't, they can pretend they don't exist. Some friends of ours put "Danger" in their kid's "other names", for example.
The Swedish situation is... confusing. :-) There are first names ("förnamn"), possibly more than one. The one in daily use — not necessarily the first one! — is called "tilltalsnamn" (something like "addressing name"), and is traditionally marked in official paperwork by underlining or with an asterisk. (You know someone has not done their homework when you get junk mail that starts with "Dear <Wrongname>!) Then there used to be middle names ("mellannamn"), which was something put between the first and last names. These were typically used e.g. by people who wanted to have both their own and their spouse's name. These are no longer issued, though those who have them can keep them. Instead, you can now have a double last name, which used to be impossible. (People have sported "double-barrelled surnames" for ages, but they have not, as far as I can understand, been officially recognised, but functioned more like "stage names").
The confusing bit is that the rules about middle names and double surnames have changed at least twice in... my lifetime, which is becoming much longer than I care to think about. :-) I didn't do a very good job of conveying that in my comment.
Small nitpick, if you are* in Germany but have an e.g. Spanish name (e.h. Hector Garcia Gonzalez) then... that's 2 surnames without a dash. I have no idea what happens if you marry or have children, but your example is just the -most basic- version.
*"you are" meaning you'd be a German citizen with a German passport.
> I have no idea what happens if you marry or have children
When you marry, you get to define what is going to be the family name and then children born from that marriage get to be registered with that family name.
In the case of foreign-born people, they can keep the naming rules from their original country. In the case of the marriage between foreigners from two different countries, you have to choose which rules are you going to follow, but the family name stays fixed.
To us (Brazilian marrying a Greek) it was a very interesting process. I have two last family names, and Greek names are gender-conjugated (i.e, the last name changes whether you are a boy or a girl). It the end the simplest thing to do was to just keep only one my last family names.
Is it that rare? I have two middle names (William Howard) and I never thought much of it. As a kid, I was more amused by a friend of my sister's who had a quadruple-barrelled surname...
I have two middle names; most financial service providers decline to recognise my second middle-name. Same goes for the taxman and my pension provider. It seems that a "full name" is no longer a canonical identifier for a person, just really a kind of nickname; the canonical identifier is now an account-number, employee-ID or whatever.
And thanks to people moving house, or even changing profession, or acquiring some defining trait or accomplishment. With most people being undocumented throughout history, names could be much more fluid, like the way a person's "nickname" may change many times with or without consent.
No. There is not limit. That can by proven by seeing people with multiple middle names or by naming your child with multiple middle names. How could there possibly be a limit? Even with the case where systems can not store or recognize the length of such a name, the middle names themselves are not limited. People's names are not entries in a compiler's symbol table.
The EU — which doesn’t include all European nations — has 24 official languages and 3 official character sets. The Danish name for the EU contains an æ (“Den Europæiske Union”). Prior heads of state of current members include Μακάριος Γ΄, with the gamma translating as “III” (i.e. “the third”). The UK has a very relaxed attitude to name changing, hence the story of Mr. Yellow-Rat Foxysquirrel Fairydiddle.
Unless Musk tries to name future kids in Emoji, European nations can probably already cope with any of this sort of thing.
There's actually an increasingly common naming pattern that I think is very weird, but in the end, I just let them do whatever. In the grand scheme of things, having a unique name can really help you stand out, and they can always go by a nickname. It doesn't hurt anything, and it could very well help.
I'm glad California regulators are focusing on the critical issue of what characters may appear in children's names. Does it ever occur to people in government that some things may not be their problem?
That's definitely a reasonable thing for governments to care about, since governments generally need to keep track of people's names in some way or other.
It's probably a good idea if government computers can keep track of the name, so a minimum requirement would probably be "can be represented by unicode glyphs". So attempts like Prince's should probably be disqualified. And even in the unicode set there are characters that may be problematic - Record Separators, Zero Width Space, Pile of Poo emoji springs to mind as examples, even if the later one might be doable. It's hard to address a letter to a person whose name just consists of a mix of different whitespace characters (especially when their neighbor is a different mix whitespace characters). So, yes, government probably should care about this, at least a little bit.
That said, Æ should probably be allowed. It's a standard character in several living languages.
The reason I think that maybe Æ should qualify is that it's not entirely unlikely to occur in immigrants' names anyway (or at least the lower case version). There are systems for transforming such characters to something that appears in passports - my own name has an umlaut, and I've obviously got a passport. I'm not American, so my passport is of course not issued in USA, but I suspect few countries would see a name diversity as great as USA so it can't be a new problem for the authorities there.
Does it occur to you that changing a database schema across an entire state government might incur significant costs and take multiple years to implement? Every courthouse, DMV, hospital, etc. might be running different, decades-old proprietary software, and who knows if the disparate, original contractors are still around.
Why is it my problem to make my children conform to government tracking and enumeration, rather than their problem that my children are hard for them to easily assimilate? Government exists for my children, not my children for government. Their technical problems are their problems, not mine.
And practically, in a world where people cross borders, where people come to this country for refuge and opportunity, how does it make sense to force them all to have only basic English characters in their names?
I'm not asking what is, but why it should be that way. Why should I name my children for the government's convenience? Why are Jürgen, Hafþór, Renée, or Noël unacceptable names?
The government(s) will be happy to receive & evaluate your generous donation of upgrades to the myriad existing record keeping systems, and further generous donations of your time to train all staff on how to enter and search for the additional written characters, so long as the process does not significantly disrupt operations. "Patches welcome!" (^o^)/
But in seriousness: it is their concern because changes that seem minor can require major changes to electronic systems, and allowing special cases that necessitate lookup in ways the electronic systems can't handle (perhaps even requring manual search through hardcopy!) really throw a star-mangled spanner in the works.
Naming is already a special case problem in Japan, where people are accustomed to having no idea how to pronounce most people's names because the parents used non-standard readings of the kanji characters and/or used kanji from a special exempted list of archaic kanji that almost nobody can read. If you're wondering why the government doesn't require people use only the official "common use" kanji, at least just for this one problem, then I should tell it has been tried — enough people raised a fuss about being unable to register their child's "perfect name" that gradually a list of "allowed only in names" kanji was created and expanded.
And if I recall correctly, when registering a name, you can specify a totally unrelated pronunciation using the simpler non-kanji phonetic characters, so even just "common use" kanji are almost "all bets off". A relative few kanji have such common pronunciation in names that they can 'usually' be guessed.
And the problem is the same with many Japanese place-names, having little or no correlation between written form and pronunciation or meaning.
So, why is the spelling of a chosen name any concern to a government? It gets crazy out there, in Name Land. How mäný variatǐons of spelliñg cån ße ællowèd before people give up on pronouncing it?
Why is everyone trying to convince me that handling names is hard for computer systems? That's obvious to everyone, even me.
What's fascinating to me is that none of you consider that government might not need to have a list of everyone it governs, or that such a list might not need to be centralized or computerized. When faced with a facet of humanity that's too complex to be easily reduced to consistent data, private groups either do their best and work with what data they can extract, or else have people handle it, with our flexible, tolerant minds. When government is involved, though, the immediate response is, "We have to force people to be less complex!"
I'm beginning to see technocracy as the biggest threat to a diverse, human, and humane society.
This is actually probably good. My wife has a hyphen in her first name and the amount of government systems including Social Security itself will most of the time result in some validation error when the hyphen is included and it even adds confusion for employees when in an office and their ancient software doesn't even result in an error, it'll just straight up not work and not give them any context on what could be wrong.
My wife and I changed our last names in California when we got married.
There are quite specific rules:
The new name must be in the format {substring of original name 1}{?-}{substring of original name 2}, or you have to go through the much more arduous and expensive full name change process (though you get a 2 for 1 discount).
> as Æ is not a letter in the modern English alphabet.
That specious, meaning appearing true but actually false. It is a diphthong expressed by a ligature of two letters. The claim was never made that a diphthong is a single letter, and it is easily expressed in modern English, in a computer using ascii and outside on a piece of paper.
In latin, æ or ae was a (di)graph for the /ae/ diphthong. In English, it is typically pronounced using the /i/ phoneme, which isn't a diphthong. I have heard it pronounced as /ai/ dipthong, but only rarely.
In English, I'm pretty sure the ae digraph is always pronounced as /i/, so calling it a dipthong would probably confuse most people.
Agreed. That is a very good point about it possibly being just a digraph and not always a diphthong. This depends, however, on the use, since it does sometimes slide and other times is just a ligature. The issues seem to stem from the pronunciation changes and the great vowel shift. As you point out, language is a messy business.
You're correct about that, sort of but not in the sense of language - only for computer definitions. A ligature in language is "a printed or written character (such as æ or ff) consisting of two or more letters or characters joined together" [1]
Need to be careful with that. There was a guy in California a few years back that decided the license plate "NULL" would be a fun joke, and he ended up being charged every ticket issued in the state where the license plate wasn't entered.
I recall pondering the mechanics of this back when I first read about it. Is some software actually replacing an input of "" (empty string) with "NULL"? Or is some comparison so loosely typed that a value of type null is considered equal enough to the string "NULL"?
'' when assigned to a char(1) becomes ' ' (char types are blank padded strings).
'' when assigned to a varchar2(1) becomes '' which is a zero length string and a zero length string is NULL in Oracle (it is no long '')
The conflation of empty string with a real null isn't great, but it doesn't also imply conflation of the string "NULL" which is what I'm trying to figure out. The four-character sequence shouldn't ever be considered something other than a four-character string, except for particular non-user-facing situations such as when actually writing code. Even spreadsheet software, which does all kinds of heuristics to find numbers in strings for example, doesn't treat the string "NULL" as anything other than a four-character string, to my knowledge.
There is no such thing as a file extension name, it's a file name extension. It's called this way because file names could only be 8-byte strings initially and then this was extended with 3 extra byte places.
Neither part of the MIME type format even has to match any existing (commonly used) file name extension anyway. E.g. it can be `text/plain`. Even when it does it is just a coincidence (although very common), it actually references the format name (IIRC `image/jpeg` was used even when almost nobody were using `jpeg` for the extension and the convention was to use `jpg`).
> * file names could only be 8-byte strings initially and then this was extended*
At least part of that is not true. The ‘popular’ MS-DOS 8.3 form derives via CP/M from DEC OSes which used 6 character file names and 3 character file types, due to their use of RAD50¹ to fit 3 characters in a 16- or 18-bit word. The type field always existed, so the word ‘extension’ most likely refers to its presentation on the end of the file name, rather than an addition to a previous format.
This is interesting because compounds are mostly written together in other languages.
Never thought about it but file extension name really is a word. Someone replied saying this is three words, but it is not is it? It's an open compound word or maybe a "set phrase", I wanted to call it an idiomatic expression but that was clearly wrong.
The Wikipedia articles for "Set phrase" [0] and "Compound (linguistics)" [1] actually don't offer much of a distinction, so it's hard to say which of those is correct. Regardless, a compound with spaces ("open," as you said) is multiple words, not one word.
> I'm a little confused about the issue description
Not helped by the fact that even with explicit cajoling* to give steps to reproduce, the reporter wrote:
> Steps to reproduce
>Try to login/create an user in Gitlab (on-premises/Gitlab.com) where the username ends with a MIME type format
Goddamit, those are not steps to reproduce! Has the current era of "social coding" and its terrible software development practices completely turned people's brains to mush?
(In fact, such cajoling shouldn't even be required. If you don't know you need to provide STR without someone going to the lengths of coddling you by going out of their way to create a template, you don't have any business using a bug tracker.)
The provided step-to-reproduce (not steps, since there's only one) seems to be regarded by the reporter as an automic operation (indivisble; a single step).
The other person is, I think, trying to express that it can potentially be done more than one way, so additional steps are required.
From a QA perspective, I prefer not to guess what the reporter might have intended. It's much better to have tons of detail, but I can sympathise with being the user and _thinking_ you have been totally clear and am know I've sometimes done this myself.
(( Unrelatedly, "ok boomer"? Since that seems such a non sequitur, I'll take in another unrelated direction and raise you an "ok athena" and "waddaya hear, starkuck?". ))
You have been trolled. The entire point of saying "ok boomer" here is to, as economically as possible, be dismissive while getting the other side to spend disproportionate energy defending the the thing being dismissed.
> From a QA perspective, I prefer not to guess what the reporter might have intended
There's not really any other perspective. That thing which you say you "prefer" is much more than a preference. It is the only reason why the bug tracker is configured to even ask for STR. If you have a "no solicitors" sign on your front door and a solicitor comes knocking anyway, or you have your door locked and a burglar climbs through a broken window and and takes all your stuff, you wouldn't respond by saying, "from my perspective, it's really disruptive and time-consuming to deal with solicitors who ignore the sign" or, "try to understand that from my perspective, it's really inconvenient when you take my things, because I have to work and spend money to replace them when I could have used that time and money doing something else." To do so tacitly legitimizes an illegitimate position held by the other.
Please look up "DARVO" to understand why explaining yourself like this is a bad idea.
This is a very confusing issue description which is caused by a confusing error message. This comment https://news.ycombinator.com/item?id=28540665 does a great job explaining the context. TLDR; you can't have a username end in .filetype because it might cause the user profile page to not load. The limitation is _not_ related to injection attack prevention, that would be concerning (bobby tables xkcd https://xkcd.com/327/ ).
And it was exacerbated by another bug which was causing the absence of a period to be ignored, so any username ending in a recognized filetype was blocked (e.g. "AsiMOV" in the example, or "MaasTIFF" in the comments).
I initially suspected that a regex was involved and someone forgot to escape the period, but it looks like that wasn't even the case -- the erroneous code was literally checking if the username ended in any recognized extension.
"ERROR: The Gitlab server has rejected the proposed username, as it ends in the same suffix as a file extension that is registered to a MIME type in the Ruby runtime under which the server runs. This list is quite long, and possibly difficult to retrieve, so we will not list it here, but you can find a list of extensions commonly used if you Google for MIME. Alternatively, if this makes no sense at all, ask a local alpha geek and they should be able to help. We understand this is weird, but the reasons for doing so are currently embargoed as they potentially have wide ranging security or stability consequences for a large number of installs. If you wouldn't mind, please do us a small one and keep quiet while we have a chance to prepare and distribute a patch without forcing anyone to forego any nights of sleep 18 months into a global pandemic complete with associated societal fracturing and potential economic collapse. Thank you for your cooperation on this easy, basic and very important matter."
It’s unrealistic to expect everyone to be able to know and use terminology perfectly. The description itself was well-written and that’s what’s important in terms of finding/fixing the issue.
It's foolish to use a standard well defined very precise but incorrect term of art like like "MIME Type" that is actively misleading to a technical audience of github users who are more likely than most people to know the standard official definition, and which is LESS commonly known than a correct widely understood vernacular term like "file name extension", if your goal is to be understood by everyone.
I'd rather a technically oriented site like github be "unrealistic" and correct and instructive, than foolish and wrong and misleading. Are you really saying it's better to use the wrong term because github users might not understand the more widely known correct term?
Just how does leading the user on a wild goose chase looking up the definition of "MIME Type," causing them to waste their time and misunderstand the error message, when it's really a file name extension (a term which more people understand anyway), help the user achieve their goals?
The bottom line is that github disallowing "MIME Types" or "file name extensions" in user names, just like a bank disallowing "select" and "drop" and "from" and "null" and "delete" and "bobby" and "tables" in passwords, is a symptom of a much larger more terrible problem, and whoever wrote that stupid error message instead of fixing the underlying bug that caused it has much worse problems than poor English language writing skills.
The bug reporter merely copies the incorrect error that gitlab gave him; The issue here is that someone working in login / security of gitlab doesn't know what a mime type is. That is extremely worrying - it's not a part of the code where you can afford to be sloppy.
It's a very odd error. Apparently .nro is a file extension used by the Nintendo Switch video game console; .o is obviously the output of compilers, so I'm not sure why my username wasn't rejected. Maybe it would be if I tried to register now.
No file extension <-> MIME type "registry" exists. File extensions exist completely outside of MIME. Many file extensions do not correspond to a registered MIME type, or in many cases even a de-facto one (other than application/octet-stream or text/plain).
It seems pretty obvious - based on the failed username in question, and to someone with fairly deep technical knowledge - that they mean anything they consider a file extension. Which is not an excuse for this marvel of awful UI slapped on top of a poorly-thought-out workaround (for some unknown vuln (that's been patched for over 2 months and is still private? quite strange for an "open" company eh?).
FWIW, the IANA media type registry [1] lists "File extension(s)" under "Additional information" for some of the media types, so it may make sense to speak of filename extensions associated with registered media types. Though given the initial odd wording, it could be anything.
Edit: as for what's actually used, looks like [2] it's the ruby mime-types gem [3], which is based on both IANA registry and various other recommendations [4], AIUI.
That "[1]" was for the registry, the extensions are listed for particular media types, such as application/xml [1]. Also mentioned in RFC 6838 section 4.12 [2].
And indeed, what IANA registers there is media types, so I've only mentioned associations. I think it still works for a-dub's argument, that it's obvious (though I'd word it differently, such as it being a reasonable guess) that if "MIME type format" is mentioned and "mov" is shown as an example, what's actually meant is filename extensions associated with registered and/or otherwise known media types (which turned out to be the case).
But "mov" is not shown as an example. The particular requestor in this issue wanted to use mov, however no example was shown in the error message. Moreover even in the requestor's case the user has to already be savvy enough to guess that "mov" is the part of "isaac.asimov" that it had a problem with, which is not at all obvious.
I don't know exactly what list GitLab is using (because, as has been mentioned, there's no central registry of file extensions), but there are many extensions more obscure and less obvious than even mov, which would further obfuscate the actual problem.
> File extensions exist completely outside of MIME.
Yes. I am well aware that certain software, such as Windows, MimeMagic, or Apache, do include their own lists. That is not a "registry" and in fact you will likely find that every such list is either identical to, a fork of, or incompatible with every other such list.
Whether you call such a list a registry or not is fairly trivial, I think. Their web server uses such a mapping, and for some reason usernames with suffixes on that list confused something. Call it a registry, or a mapping, or a list, the meaning of the statement is the same, and I don't think it prevents most people from understanding what's going on.
registry (noun): a place or office where registers or records are kept; an official list or register.
In this case, a registry would be a mapping, however a mapping is not a registry. A registry is the single, centralized place from which you can look up registrations, to guarantee that there are no conflicting records. Literally none of that is (or can be, at this point) true of file extensions.
Gitlab recently exchanged the "WIP" prefix for merge requests (Work in Progress = started to do something but didn't complete it yet) for "Draft", which has connotations of throwing the draft/sketch away to build the final product.
Which is definitively not what is meant there. But I think it shows that Gitlab is not a company I'd go to if I wanted linguistic precision.
I am not a native English speaker (and I don't work at Gitlab) but I am curious and interested in using proper terms, as I think as developer that naming is a very important skill.
So, I find strange this meaning that you give for a "draft" = that it is something that has connotation of throwing away when building the final product.
I think you are confusing a "draft" with a "sketch" and they are not the same.
I googled the term "draft" and here is what I found:
> "a version of something (such as a document) that you make before you make the final version" [1]
> "A preliminary version of a piece of writing." [2]
While "sketch" means:
> "a rough drawing representing the chief features of an object or scene and often made as a preliminary study" [3]
> "A rough or unfinished version of any creative work." [3]
> "A rough or unfinished drawing or painting, often made to assist in making a more finished picture." [4]
In the case of sketch I see some keywords like "preliminary study" or "assist" or "unfinished version" that indicates that the sketch will not be the final product.
So while it is true that a Draft could be thrown away if someone has new/better/difference ideas while working on it, it does not seem to imply that a Draft should be thrown when building the final product.
As far as I understand it is more that a draft will evolve into a final product or might be abandoned.
Unfortunately, here, I think the problem is that English is not very clear.
As a native English speaker, the first time I encountered the word 'draft' was at primary school. We used it to describe a piece of writing where presentation was not the focus, instead, content and accuracy in terms of spelling, grammar, and punctuation would be the focus. Once the drafts were complete, we would 'copy these up' in our neatest handwriting.
However, if I prepare a 'draft' of some document or other for my boss, I expect it to be essentially an unapproved version of a final document, perhaps needing some minor modifications before release, but also perhaps not.
Although personally, I would use 'sketch' to mean something disposable which illustrates a more perfect version, my grandmother is an artist, and she refers to the initial drawings she makes on the canvas as 'sketches', which she then paints over in more detail.
Essentially, I don't think there's a big difference between draft and sketch - both could (in my opinion) represent either a version which will be discarded or which will be developed further.
Overall, I think here, the 'Work In Progress' label is the clearest and least likely to be interpreted differently by different users.
It's also interesting the in English we also have Drafters or draughtsman/draughtswoman, who's job is drafting, which is the process of creating technical drawings for manufacture.[1]
In a highly technical environment this is the kind of work I associate with drafting, but not necessarily with the word draft.
The wikipedia page for "Draft" has a veritable smorgasbord of different things that are considered "Drafts"[2], which kind of illustrates that getting pedantic about the definition of the word is losing proposition.
But a native english speakers as well, I agree with you, that the everyday colloquial definition of the word "draft" is an incomplete piece of work that needs further refinement before it can be considered complete or final.
> I am not a native English speaker (and I don't work at Gitlab) but I am curious and interested in using proper terms, as I think as developer that naming is a very important skill.
That is it for me too and why I think this change is so annoying.
WIP means something is being worked on. [0]
Draft can mean the same, but it also has a bunch of other possible meanings. Note that if you search in both of your sources, you'll find "sketch" as an explaination. It can mean the same, but it can also mean a bunch of other things. It's a strictly worse name.
Most likely the rename is in order to get more people to understand what it is without having to look up abbreviations. "Draft" is clear to most people who know English, while "WIP" puts a lot more burden on the reader in terms of what they already need to know in order to understand.
A draft in the visual arts (drawing, painting), is typically abandoned, like a sketch. Perhaps it uses a different medium to the final version, and in any case can't easily be adapted.
A draft in writing is typically incrementally improved, or at least we think of it that way now that we write on computers. We don't need to start again even for substantial changes like adding a new paragraph.
In internet, in most places I've seen "draft" as being something that is not yet published. For example in a lot of blogging software like WordPress, you can save your post and it will be "Draft", but only will be visible to others when you publish, so it's not thrown away in most contexts that I know it being used in the Internet.
Isn't it the same connotation here? The work is shared with your collaborators, but not yet "published" to the master branch or to customers or wherever?
> which has connotations of throwing the draft/sketch away to build the final product
I think it's only software developers who urge others to first write a draft, then throw it away and start working on the real thing. Usually, a draft precedes the "real" version and it's a status attached to something. Eventually, a "draft" becomes "published" or something similar.
Authors, scientists, email writes, report creators, movie/music producers all create drafts that (maybe) eventually become the real thing, I don't think many of them throw away the draft but rather work on the draft until it's not a draft anymore.
> it seems pretty obvious that they mean any file extension registered to a known MIME type.
I guess I'm dense, but I actually thought it's about users such as Mr. Joe R Text/Plain. When I read about "mov" in the actual issue, then it became clear.
Since file extensions can be any three or fewer (or even more) valid characters, then no string ending with zero to three characters, in other words, no user names are valid.
This seems on par with the general GitLab-style. Is anybody else getting a bit frustrated with them?
They keep on having high-severity security bugs being fixed every month (e.g. auth checks not being done everywhere). Then there's all these odd edge case bugs everywhere.
As an outsider, it just seems to me that GitLab isn't being engineered in a principled way: on sound abstractions and with separation of concerns (e.g. auth should be some universal middleware, not ad-hoc per call). Just really basic stuff.
GitLab team member here. I'd like to add some additional context to my previous comments [1][2].
Due to a security concern in which a profile containing a file extension would not load [3], we do not allow usernames that end with file extensions (ex: .mov). As noted by many folks here, these are associated with a MIME type but are not MIME types themselves. It is not related to preventing an injection or any such attack vector.
The error message for this check incorrectly included MIME type rather than file extension. This has been updated [4].
Additionally, there was an issue with how the actual check as it did not include the leading dot. The leading dot was added to the check in a subsequent MR [5].
>Due to a security concern in which a profile containing a file extension would not load [3], we do not allow usernames that end with file extensions (ex: .mov).
Why did you not fix your routing engine to not consider file extensions where the username / group name should go?
gitlab profile URLs are `gitlab.com/<username>`, so a user with the name "dashboard.html" would have the URL `gitlab.com/dashboard.html`, which obviously conflicts with the existing dashboard.html file.
Besides blacklisting certain usernames or breaking a bunch links to profiles, how would you fix that?
EDIT: IIRC, github has the same issue, but they have profiles as lower priority. So if your username conflicts with an existing URL, your profile page doesn't work.
Isn't the problem arising because GitLabs files are in the global namespace? If the user is the namespace for all their files, and GitLab files were under a Gitlab user, this wouldn't be a problem. Under the current implementation, every time you add a file, you have to make sure its name doesn't conflict with an existing profile. And a username has to avoid conflict with all present and future filenames. Mutual pain doesn't seem like a good way forward.
3. Change the URL for _all_ usernames to `gitlab.com/u/<username>` as this collision points out the flaw in the original URL design in the first place, because of possible collisions. 301 redirects could of course be used for any non-colliding usernames.
I am now wondering how _github_ takes care of it though. Github also has `github.com/<username>` urls. What does it do with collisions? Github pages don't even all end in `.html` or contain a `.` at all, so gitlab's particular solution would not work. For instance, there is a page `https://github.com/topics`. What happens if you try to create a github user called "topics"?
If I try to create one, it says "Username 'topics' is unavailable." Same for say `marketplace` or `trending`. Perhaps they've deny-listed only actually-existing github urls? That does seem tricky, whenever they want to create a new top-level /page on github, they can only do it if there isn't already a github account with that name?
But if as someone else says `/dashboard.html` is just a weird non-canonical alternate for `/dashboard`, which already had to be reserved, maybe gitlab is already doing (1) anyway? Then why do they need to also deny any username with ending in any valid extension? Unclear.
It still makes me wonder if they have a routing precedence problem, which they worked around by just forbidding any username that triggered it, instead of fixing the actual issue.
In what situation would someone be requesting `https://gitlab.com/dashboard.html`? When I go there, I get the exact same page as I get at `https://gitlab.com/`, why was it necessary to support both URLs? Now they're stuck with it of course, if anyone actually uses /dashboard.html, but surely they could just special-case filenames that actually exist, just like they presumably special-case URLs they use like /help already. It doesn't seem necessary to blanket-ban anything with a file extension.
Or gitlab.com/dashboard. GitLab, GitHub, etc already have a need to reserve specific usernames (like `org`, `settings`, `projects`, `new`, `explore`, `marketplace`, `help`, ...). Since you already have to blocklist specific names not containing extensions, I really don't see how banning extensions help them.
Hopefully we'll know more once their security ticket becomes public.
and still have it return a proper HTML document that covers that user's profile page. Hell, the username could be some insane zalgo-tier shit and still function properly.
I see some comments defending arbitrary "bandaid" architecture and I think that this is not defensible for something the scale of GitLab. This is basic HTTP stuff.
Of note, GitHub doesn't allow periods in usernames either. I'm not a Ruby expert but I wonder if file extensions give some specific Ruby gotchas that means both GitLab and GitHub operate this way.
It's not a Ruby gotcha, it's a Rails routing gotcha. You can specify alternative formats on any path, e.g. you can access /path to get HTML (or whatever the "default" format for that controller method is) or you can access /path.json or /path.xml and if the controller method specifies handlers for those formats, you get that format. So if you allow a username like "john.doe" and the route is something like /john.doe then Rails will interpret the "john" as the ID part of the path and "doe" as the format part of the path. You can override this in your routes to support periods but then you do lose the capability of accessing alternative formats which sometimes can be useful.
Thanks. Now I also get a better understanding of why .patch for MRs or .keys for user names as file extension work on both Rails platforms. I always found these file extension hacks very useful for quick access and automation.
I could imagine the mix of rails #respond_to and "file extensions" at the end of urls might make a mess (think /users/profile/smith.html vs /smith vs /smith.json vs /smith.txt - essentially what might have been /smith?format=json etc).
Once had my Australian production system go down because a js plugin we were using had added .au to its list of media types. Integration had a different TLD.
Response I got from the author was 'LOL!'.
:)
Every programming language has a library to parse urls.
Please, for the love of your own mothers, people: stop pretending like you know how to parse urls like they're strings. You don't. And even if you can, you won't do it right every time.
It's been demonstrated over and over again.
Just use the already tested url facilities to give you the host/path/query parameters
Probably: some web frameworks do content negotiation by appending a content type like .json to the end of the url
Not sure if it's an attack vector per se, or just that the behaviour is incompatible with allowing usernames containing . and then having urls where the username is the last segment of the url
For non-programmatic usage and verification, the extensions can be easier?
On a similar topic HIBP allowed people to request versioning via a custom HTTP Header, a Accept Content-Type, or a version segment in the URL path and approximately everyone went with option 3.
"For non-programmatic usage and verification," it should be served in a human-readable manner, typically including things like formatting, and maybe even syntax coloring if you're feeling nice.
Take my `Accept: text/html` and give me my HTML-ified JSON, dangit! ;)
I've actually seen APIs with "pretty=true" or "indent=4" type query parameters to emit formatted JSON for people before, though it doesn't go as far as HTML with syntax colouring.
Yes, the content negotiation you describe is a very longstanding default behavior of Rails. It should probably be made opt-in rather than opt-out in the next major version.
Somebody probably put in a regexp with .mov$ , however for regexps the dot (.) matches everything (and $ matches end) so the i in asimov is eaten regardless and then the rest of the match succeeds.
'The problem was named after an incident in 1996 in which AOL's profanity filter prevented residents of the town of Scunthorpe, Lincolnshire, England, from creating accounts with AOL, because the town's name contains the substring "cunt".'
>'The problem was named after an incident in 1996 in which AOL's profanity filter prevented residents of the town of Scunthorpe, Lincolnshire, England, from creating accounts with AOL, because the town's name contains the substring "cunt".'
Right. Regardless of the specific pattern matching function, in both cases, the results were both incorrect and unwanted. Which is why I consider this instance to be a variation on the same issue.
- In the issue comments the Gitlab employee says "Sorry, I cannot go into details right now. I will link the issue here once it goes public, is it ok?"
It could maybe potentially be exploited in a very interestingly crafted email, where there's link to download something (e.g. the source tarball, or a build artifact) with an URL containing the username, or being otherwise close by, so that the downloaded file would be interpreted differently. But I'm not creative enough at this hour to suggest a working exploit.
Maybe it's something to do with a MIME sniffing attack. The user profile URL may be detected as a different MIME type by the browser based on the extension: https://gitlab.com/myname.js
AFAIK in MIME sniffing IE guessed file type based on its content [1], not URL. And given that IE is rarely used nowadays not sure if it still relevant.
I've played with this before. A correctly implemented mail library should handle e.g., subject lines that contain SMTP control characters. I developed a lengthy repro for an email parsing issue in an ancient version of some java email library that contained a truely horrendous parser, only to find out that the library had been updated internally recently :-)
Even webserver returns a mime, browser may sometimes still ignore it and guess it themselves, because server may be wrong. Unless the server ask them `absolutely don't do it` via the header. a.k.a. `X-Content-Type-Options: nosniff`
But even there is a header for that, there will still be situations that clients don't care and guess it themselves (probably because software is too old or other reason).
Correct. You want to set Content-Type and X-Content-Type-Options.
The story is that in some cases, Internet Explorer could be tricked to ignore the Content-Type header. Instead of fixing the core bug, Microsoft decided to add a header (presumably, they didn’t want to break existing websites). A decade later, people discovered two ways to bypass X-Content-Type-Options. This time, Microsoft fixed the core issue instead of adding X-Content-Type-Options-For-Reals.
Actually when I reported security issues with XCTO:nosniff to Microsoft the answer was something like "well... we don't care, but you can hope that with Edge going to chromium engine things will change". Firefox fixed it eventually.
I think even browser nowadays still do it. A file in <script src="" /> will be treated as script even mime is wrong. A file ended with .mp4 will be play as video even it says it is a text/plain file. Browser guess mime from contents at many places, just you may not notice it.
There is not much ruby magic in that code. This is just a naive use of data from another module.
Somewhere in the Gitlab code base there is a MIME_TYPES map with common extensions as the map key. No idea what it is used for but that module is very likely the target of a recent security issue.
The first fix to combat the "publicly unknown" vulnerability was to prevent usernames ending with any of the keys in the MIME_TYPES map using a simple "ends_with" strings check. Of course the map keys did not have periods so the ends_with would also match "Asimov" with the "mov" suffix.
The second fix in this PR is to extend the ends_with check to add an extra dot.
The actual vulnerability is still unknown but I suspect it's something like an intermediate component that performs special handling based on interpreting URLs and that could bypass security/ACL checks.
Thanks for the link -- you solved a nagging mystery for me[0]. In the '90s, I worked for a tech guy from Texas who referred to the clippers that we used to cut the ends of zip ties off as dikes.
Of course, being a teenager, the term often came with a raised eyebrow from me since it was commonly used as a slur for lesbians. He was the only person I've encountered in my life who used that term in that manner and over the years I just assumed it was some bit of obscurity related to where he started working in systems. I guess not!
[0] I'm sure the right Gooble query would have gotten me an answer but it teetered on the edge of "I don't care enough to bother" until the answer was presented.
That's what actual MIME types are for. You can serve a .jpg as a .txt file and still have the browser display it correctly as an image if you use image/jpeg. Same for the other way around or any other combination of file and MIME types.
You're not wrong, but I bet there are lots of things that will assume based on url (rightly or wrongly). e.g. I've seen a lot of nginx reverse proxy setups implement different caching policies based on url suffix (again rightly or wrongly).
Yeah, if you for example use the 1001th hip electron based browser (with some hand-crafted curl workers and custom fetchers with an express based proxy-backend included in the background mangled together with some weird mix of 20% javascript and 80% typescript and a combination of at least 400 different npm packages over a commit history of two weekends) to browse the web ... of course, that assumption might break. :^)
Do you remember when MSIE would ignore the MIME type the server sent, and sniffed the contents of the beginning of the file downloaded and overrode what the server sent?
> Staff member made a change to disallow usernames ending with `Mime::EXTENSION_LOOKUP.keys`
This is a hell of a thing to commit and pass code review in 2021 on a project like GitLab. I understand that the staff member was fixing a security issue and was probably not thinking deeply about the ramifications, but even so. How many "Falsehoods programmers believe about names" articles do we need?
I don't want to be too harsh on the programmer, everyone makes mistakes. It's easy to see how a person focused on a security issue with user names makes a quick fix without thinking through how this will affect account creation. As programmers, we have to make like 7,000 decisions every day, you're bound to fuck up some of them. This is a pretty big one, though.
The bigger question is how this passed code review and testing.
That's a good question, but I think it's still more important to understand why a root cause analysis wasn't finished. They got the cause, but not the root cause. It's like seeing your car battery dead and just replacing the battery, without understanding whether the alternator works or whether the alternator belt is broken.
What's the security issue though? Not disambiguating filenames and user names in URLs for the site? That's a hard problem to fix after the fact, which is why we call it a Rookie Mistake.
Use unique prefixes for users, groups, projects, assets in your webapp design, kids.
Surely it would be easy to run the new rules against random-list-of-usernames-found-through-google before pushing to prod? Or perhaps the security issue was deemed so great that they needed a fix out yesterday?
If anyone ever says that software engineering is completely unlike plumbing, I will point them towards your comment. All systems, however nicely architected, are full of duct-tape solutions like this.
As someone who has traveled to some of the lesser traveled places on this planet, they do too! Sometimes even worse techniques are being used to fix things.
Are they though? What I can think of (broken upstream reverse proxies that do mime type inference by filename) would warrant a WE_USE_BROKEN_LEGACY_SHIT_UPSTREAM config flag so that it doesn't get in the way of normal users.
So I'm probably missing something and I'm really curious for the underlying vulnerability.
I have no clue about the underlying issue, but I'm guessing it's occurring on a boundary or an interplay between two systems.
Something like "the username can be part of the URL, and if the URL contains .mov, some browsers will misinterpret this and assume it's a movie file, leading to bad things™".
Or: "the username is sometimes used as a folder name, and our syncing software contains rules to exclude certain file extensions, so these folders were never synced, which lead to issues on production servers"
I'm guessing it's something along these lines. Something that you control, but not really, leading to these kind of haphazard workarounds.
That seems to be a consequence of what IMO is an unfortunately common bad design: having user-controlled data like usernames as the first path component (without a prefix like ~). There are many things which are expected to be found at the root of the path (the classic example being robots.txt, but there's also favicon.ico, .well-known, and probably others; I vaguely recall that IIRC Flash used a fixed filename in the root for cross-domain access control), and you never know when a new one will be invented by someone (though .well-known is supposed to contain the spread of these "magic" names).
Don't know the details of the vulnerability, but from the comfort of my armchair, it sounds like it's being patched in the wrong location.
e.g. It's better to fix an XSS issue by escaping the input, rather than restricting the values it can take.
Further proof that gitlab is just super jank. What on earth are they doing inspecting usernames to see if they end in a file extension? What horrendous vulnerability is this band-aiding?
That they don't seem to understand what a MIME type is just adds to this perception.
It's not that weird if you look into it. Imagine a username "index.html", so the URL to their profile is `gitlab.com/index.html`.
Github has the exact same issue, and they solved it by just restricting usernames to alphanumeric characters and hyphens (but IIRC, there are some existing profiles with unreachable URLs from before they made the change).
Gitlab decided to just restrict known file extensions, which IMO is not the best idea, but it's also not that unreasonable.
> That they don't seem to understand what a MIME type is just adds to this perception.
That's not fair, people misspeak/mistype things all the time.
What would be the problem with gitlab.com/index.html username? There's no rule that a web server must serve up index.html from some /public/ folder just because that's the URL. This screams poor engineering to me.
I am in complete agreement with this statement. Unless they're running on an ancient LAMP stack or something the web server isn't going to do anything with the .html extension (or any other extension), and browsers are supposed to look at mime type.
Yeah, this didn't leave me with a good impression of their security and development practices (not that I really had one anyway after they accidentally deleted their production PostgreSQL data directory and did a fairly poor job of responding to the situation).
This change absolutely seems like the wrong place to fix any real security vulnerability and the fact that it affected a bunch of legitimate usernames is the icing on the cake.
I've never used gitlab but this issue has definitely caused me to take pause. It sort of feels like "not allowing 'OR' in strings to prevent SQL injection."
Hi, the error message is a bit confusing - certain file type extensions may cause the user profile page not to load. Disallowing specific extensions in the username helps prevent that problem - it is not to prevent an injection attack or similar.
> certain file type extensions may cause the user profile page not to load
There is no reason for this. They should always hit the same controller and be served as text/html. Why would the username ever influence this?
If that breaks an existing page then you either shouldn't have allowed the username to be created with the same name as an existing page (exactly what GitHub does) or shouldn't have created a page with the same name as a username [depending on which came first!] - or better yet, had them in a separate namespace so they cannot conflict in the first place.
That you change the Content-Type of a page based on the URL is quite a vulnerability on itself no?
I don't think browsers infer meaning about file types based on the URL. The Content-Type is always what is being used.
If you have backend side code that maps URLs Content-Type header mime types. Don't. Instead simply always return text/html for user profiles. Then the extension shouldn't matter.
Rails, which GitLab is based on, uses suffixes to select content renderer (such as .html, .json - apparently in case sensitive manner), also I think it will serve underlying files in some cases (e.g. the example of dashboard.html) given elsewhere.
The proper fix is to disable this mechanism at least for the username segment of gitlab path but perhaps GitLab developers are too lazy or unaware or just in rush.
The fact that usernames can break anything speaks to something profoundly wrong in general. Why should usernames be leaking into URLs instead of a surrogate key?
I mean, barring some tricks that could break layouts during rendering if people got really creative with them, why isn't any valid non-blank Unicode string a valid username?
gitlab naming restrictions are also ridiculous for repositories.
For example our files repository needed to be named filez.
There is another repo which we needed to rename while migrating to gitlab but I dont remember it's name now. This has made many issues for us because repo name is no more related to project name...
I suspect because Isaac Asimov was named before any of these problems with his name were a thing and his name is not something artificially constructed to cause problems with buggy computer systems.
Sounds like a WAF is getting in the way. Wikileaks had the same issue years ago when you couldn't search for programming language binaries e.g "Ruby" or "Perl" etc
Basically something to extract boatloads of money from enterprise customers by annoying THEIR customers so they can't write "<script>" in texts in their application.
Or, less tongue-in-cheek, a way to harden web applications against known attack patterns like sql injections or xss-attacks. As they work on pattern recognition and don't know anything about your application they sometimes get in the way. But they'll probably check some box for some security audit so they're used.
oh weird. i thought they also did stuff like parse and reconstruct requests to try and catch any funny business and centralize/add ease for things like ratelimiting and fail2ban for webapps. looking at this one, it appears not.
Examples of MIME types: "text/plain", "text/html", "image/png" "application/pdf", "video/quicktime", ...
If I was prevented from using the username "wcoenentext/html", then I wouldn't really be bothered by that. (Although I might question the design decisions that would necessitate such a restriction.)