Your Grandfather's birth is registered in Wallingford in 1911 (vol 2C p655), with mother's maiden name as Vaughan.
Henry J Osborne married Mary Osborne in Wallingford in 1910.
A search of the 1911 census finds Henry and Mary living with her mother , your grandfather is not shown on the census, he was born later in the year. Both Henry and Mary are shown aged 27, so born about 1884. The dates you give suggest you have missed a generation.
There was a Henry J Osborne who married Nellie Bennellick in Easthampstead in Oct-Dec 1927 and they had a child Laura in Jan-Mar 1928. If this is your grandfather, he would have been just 16yrs old - and Nellie would have been heavily pregnant, so it could have been a "rushed" marriage. Getting a copy of the marriage certificate for Henry & Nellie would give his father's name and occupation and would help check if this is actually him. If it is , the next line of enquiry would be to trace the daughter Laura, if still alive, or her descendants, who may know more about the family story.