The purpose of an Annual General Meeting (AGM) is to elect a new board of directors and to present the financial results of the past year to the stakeholders. Has your organization defined what a successful AGM would look like? Is there an attendance number that would cause you to declare victory?
Low attendance could indicate that your organization has not attracted much attention from stakeholders or it could simply mean that the stakeholders do not feel the need to come out to a meeting. If your group wants to have a lot of people attend an AGM then there might need to be a bribe, such as food or an interesting speaker. If success is going to be defined by attendance, then setting the date early and starting to market the AGM is a good strategy to improve attendance.