In geometry problems square brackets denote area. So $[ABCDEF]$ denotes the area of polygon $ABCDEF$.