One is a very, very special number. It has properties that no other number has. When multiplied, it gives the same answer. When divided, the number remains the same. Every other number is made up out of ones grouped together. The square root of one is one. It is not always a prime. It is definitely not a composite number. It stands for a single unit (from which we get the term "unity") and represents wholeness. One is a creator, not the created, and is thus something that is unique in numerology. It was not until the late 1700s that it was decided that one was indeed a number and not a special Unit that acted like a number.
An expert from Mathematicks Made Easie: Or a Compleat Mathematical Dictionary, Explaining All the Parts of the Mathematicks, with All the Terms of Art, and Difficult Phrases Rendred Plain and Easie to Every Capacity. Collected from Monsieur Ozanam's Dictionaire Mathematique, Vitalis, and Others; with an Appendix Containing the Quantities of All Sorts of Weights and Measures, the Characters and Meaning of the Marks and Symbols, Or Abbreviations Commonly Used in Algebra. Also the Definition, Explanation, Nature and Meaning of the Principal Mathematical Instruments, Illustrated on Copper Cuts Curiously Engraven. By J. Moxon at the Atlas in Warwick-Lane, and Tho. Tuttel Mathematical Instrument-maker to the King's Most Excellent Majesty, at the King's Arms and Globe at Charing-Cross, and Against the Royal Exchange in Cornhil
Even at the very beginnings of counting one was unique. Indo-European actually had "one" as a gendered (inanimate vs. animate; one was considered animate as it ended in -s) word that could could decline. Indeed, that is what it was used for more-so than counting. The word, *sems, is the root of such words as same and once. Another word was also developed, *oi-, which had a suffix in most cases that caused it to end in -os, which places it squarely in animate gender with the other form. This word has descendants such as single and one. Two is not the same, descending from *duwo, which had a feminine and neuter form as well (*duwoi / *dwoi) as do three and four (five and beyond do not have declensions). What the evidence tells us with these forms and meanings is that initially "one" was not where counting started. Singular existed, plural existed, then numbers beyond simply "plural" were created but one was not a necessary number word. (Numbers, Numerals, and Count in Indo-European).
This strangeness of one as an origin of number but not number can still be seen in English today. Consider this phrase: "I have a number of crowns". Would this sentence work with zero or one? No. The sentence becomes almost nonsensical. The phrase "a number" certainly implies more than one or two. "A small number of people" would almost never be one and could not be zero. In line with the Indo-European system of singular and plural we have "a" and "a couple", "a few" and "several" that all have distinct meanings with lower boundaries.