There is no James (or Robert) Firby/Furby in the 108 records I have.
There is a James birth 23 MAY 1833 on the IGI, however it has been submitted by a member of the LDS Church. You may be aware these submissions are not always reliable. Another James birth in 1833 but parents John and Elizabeth so not yours.
The marriages of Robert and John are on the IGI. I am sure you will have seen them.
I have tried searching for this family in the England 1841 census without luck. Perhaps they had emigrated to Canada by then?
Sorry I can't help further.