Early aircraft had multiple crew. Asides from a pilot/co-pilot position, they also had a navigator, radio-operator, mechanics (servicing the engines in-flight and carrying spare parts due to low reliability), and flight engineers. One by one each of these positions has been made obsolete due to improvements in automation and aircraft reliability. The aircraft safety record has only improved over time, so eliminating aircrew has not affected safety.
One man commercial flight operation is already allowed for small aircraft with under 20 passengers and has not been seen to adversely affect safety. For short haul flights, a 2nd pilot is only needed as a backup, and modern auto-flight systems are reliable enough to handle the small chance of a pilot incapacitated mid-flight.