Most of the phones allow direct IP calls. Many of the phones allow assigning IP addresses to speed dial buttons allowing dialing with a single push - just make sure phones have static IP addresses and add labels to speed dial. You can also setup multicast RTP streaming for a one-way broadcast from a phone to all other phones or some group.

For a few more features (and some reliability loss) you can either buy some cheap new/used small PABX or register phones to some online service (there are free ones if you would be calling only “inside” this provider, though there might be privacy concerns).

Recommendations are not allowed here, but before you spend money - there are at least 3 free options (1 x Linux + 2 x Windows).

Headset/headphones amplifier levels might vary. You might need external preamplifier. Maybe some cheap LM386 module. Alternatively you can try using speaker ouput as audio source, this might need audio transformer though, class-D audio amplifier in PA-1 is probably full-bridge.