Five things that modern science cannot explain:

[mathematic]

72. A patent office administrator said that about inventions, not science

And he didn't even say it!

"Everything that can be invented has been invented" has been attributed to Charles Holland Duell but he never actually said that, or anything like that.

Physicists and other scientists were well aware of the gaps in scientific knowledge in 1900. In particular, in the example you refer to, the photoelectric effect was discovered in 1887 and explained in 1905 (by Einstein). In physics, the late 1800s were about working out the consequences of Maxwell's unifying work on thermodynamics (1865).

1900 also saw Hilbert's problems, the famous and influential list of 23 unsolved problems in mathematics. One called for the axiomization of physics. Much of 20th century mathematics can trace back to solving these problems. Incidentally, Hilbert was either an independent discoverer or co-discoverer of general relativity. Mathematicians like to give him joint credit with Einstein.