The yogh descended from Latin g, but it is not part of any present-day Latin alphabet. In times when the computer Latin alphabet (codepage) was restricted to some 200 characters, they only accepted letters still in use.

The thorn, indeed, originated from the Greek theta (via Italic and runic), which the ancient Romans had transcribed th because of its pronunciation in some Greek dialects then. Thus, M, you are right that the thorn does not come from ancient Roman. But in current Icelandic language, it is indeed part of the (otherwise) Latin alphabet, towards the end.