Ellison was born on August 17, 1944 in New York City to Florence Spellman, a 19-year-old unwed mother who later placed her nine-month old son for adoption to her distant relatives. Lillian and Louis Ellison took him into their home, a two-bedroom apartment located in a modest lower middle class Jewish neighborhood in South Chicago. Ellison recalled to an interviewer that he had a warm and loving mother opposite to an austere and unsupportive father. At South Shore High School, he was a bright but inattentive student. At 15, he began a long-term relationship that lasted for five years and ended sorely – depending on whom is asked, he unsuccessfully proposed marriage either once or twice. He lasted until the end of his sophomore year at University of Illinois at Urbana-Champaign but dropped out following Lillian's death. After a summer in Northern California, he returned home to study at the University of Chicago but left after one quarter. Ending his attempts to finish college, he set out for California.
During the 1970s, Ellison worked for the Ampex Corporation. One of his projects was a database for the CIA, which he named "Oracle".
Ellison was inspired by the paper written by Edgar F. Codd on relational database systems named A Relational Model of Data for Large Shared Data Banks. He founded Oracle in 1977, putting up $2000 of his own money, under the name Software Development Laboratories (or SDL). In 1979 the company was renamed Relational Software Inc., later to be renamed Oracle after the flagship product Oracle database. He had heard about the IBM System R database, also based on Codd's theories, and wanted Oracle to be compatible with it, but IBM stopped this by keeping the error codes for their DBMS secret. The initial release of Oracle was Oracle 2, even though there was no Oracle 1. The release number was intended to imply that all of the bugs had been worked out of an earlier version.