Voting Common Stock [Member] Nonvoting Common Stock [Member] Class of Stock [Axis] Statement of Financial Position [Abstract] Cash and due from banks Short-term investments Total cash and cash equivalents Cash and cash equivalents, beginning of period Cash and cash equivalents, end of period Available-for-sale securities, at fair value Fair Value Securities available-for-sale Residential real estate loans held for sale SBA loans held for sale Total loans held for sale Loans Total Loans Total-loan Less: Allowance for loan losses Beginning balance Ending balance Total Loans, net Premises and equipment, net Real estate owned and other repossessed collateral, net Real estate owned and other repossessed collateral Federal Home Loan Bank stock, at cost Federal Home Loan Bank stock Intangible assets, net Bank owned life insurance Other assets Total assets Demand Savings and interest checking Money market Time Total deposits Federal Home Loan Bank advances Wholesale repurchase agreements Short-term borrowings Junior subordinated debentures issued to affiliated trusts Capital lease obligation Other liabilities Total liabilities Commitments and contingencies Preferred stock, $1.00 par value, 1,000,000 shares authorized; no shares issued and outstanding at September 30, 2015 and June 30, 2015 Preferred stock, par value (in dollars per share) (in Dollars per share) Preferred stock, shares authorized Preferred stock, shared issued Preferred stock, shares outstanding Common stock Common stock, par value (in dollars per share) (in Dollars per share) Common stock, shares authorized Common stock, shares issued Common stock, outstanding Additional paid-in capital Retained earnings Accumulated other comprehensive loss Net-of-tax amount Total stockholders' equity Balance Balance Total liabilities and stockholders' equity Statement of Cash Flows [Abstract] Interest and fees on loans Interest on available-for-sale securities Other interest and dividend income Total interest and dividend income Deposits Federal Home Loan Bank advances Wholesale repurchase agreements Short-term borrowings Junior subordinated debentures issued to affiliated trusts Obligation under capital lease agreements Total interest expense Net interest and dividend income before provision for loan losses Provision for loan losses Provision Net interest and dividend income after provision for loan losses Fees for other services to customers Gain on sales of residential loans held for sale Gain on sales of residential loans held for sale, net Gain on sales of portfolio loans Gain on sales of portfolio loans, net Loss recognized on real estate owned and other repossessed collateral, net Loss (gain) on sale and impairment of real estate owned and other repossessed collateral, net Bank-owned life insurance income Bank-owned life insurance income, net Other noninterest income Total noninterest income Salaries and employee benefits Occupancy and equipment expense Professional fees Data processing fees Marketing expense Loan acquisition and collection expense FDIC insurance premiums Intangible asset amortization Amortization of intangible assets Other noninterest expense Total noninterest expense Income before income tax expense Income tax expense Net income Net income Net income Basic (in Shares) Weighted average shares used in calculation of basic EPS Diluted (in Shares) Weighted average shares used in calculation of diluted EPS Basic (in Dollars per share) Basic Diluted (in Dollars per share) Diluted Cash dividends declared per common share (in Dollars per share) Dividends per share Change in net unrealized gain (loss) on available-for-sale securities Change in net unrealized gain or loss on available-for-sale securities Total available-for-sale securities Change in accumulated loss on effective cash flow hedges Reclassification adjustments for net gains included in net income Reclassification adjustment for net gains included in net income Total derivatives and hedging activities Total other comprehensive loss, before tax Income tax benefit related to other comprehensive loss Other comprehensive loss, net of tax Other comprehensive loss, net of tax Comprehensive income Statement of Stockholders' Equity [Abstract] Balance (in Shares) Balance (in Shares) Common stock repurchased Common stock repurchased (in Shares) Dividends on common stock Stock-based compensation Issuance of restricted common stock Issuance of restricted common stock (in Shares) Forfeiture of restricted common stock Forfeiture of restricted common stock (in Shares) Conversion of voting common stock to non-voting common stock Conversion of voting common stock to non-voting common stock (in Shares) Equity Components [Axis] Preferred Stock [Member] Common Stock [Member] Additional Paid-in Capital [Member] Retained Earnings [Member] AOCI Attributable to Parent [Member] Balance Sheet Location [Axis] Loans Location [Member] Accretion of fair value adjustments Deposits Location [Member] Borrowing Location [Member] Originations of residential loans held for sale Net proceeds from sales of residential loans held for sale Depreciation of premises and equipment Loss on sale and disposal of premises and equipment, net Stock-based compensation Amortization of available-for-sale securities, net Other assets Other liabilities Net cash (used in) provided by operating activities Purchases of available-for-sale securities Proceeds from maturities and principal payments on available-for-sale securities Loan purchases Proceeds from sales of portfolio loans Proceeds from Sale of Loans Held-for-investment Loan originations, principal collections, and purchased loan paydowns, net Purchases and disposals of premises and equipment, net Proceeds from sales of real estate owned and other repossessed collateral Net cash used in investing activities Net increase in deposits Net increase in short-term borrowings Repurchase of common stock Dividends paid on common stock Repayment of wholesale repurchase agreements Repayment of capital lease obligation Net cash provided by financing activities Net (decrease) increase in cash and cash equivalents Transfers from loans to real estate owned and other repossessed collateral Disclosure Text Block [Abstract] New Accounting Pronouncements and Changes in Accounting Principles [Abstract] Investments, Debt and Equity Securities [Abstract] Securities Available-for-sale [Abstract] Major Types of Debt and Equity Securities [Axis] US Government Agencies Debt Securities [Member] Amorized Cost Gross Unrealized Gains Gross Unrealized Losses Collateralized Mortgage Backed Securities [Member] Collateralized Debt Obligations [Member] Summary of Gross Unrealized Losses and Fair Values Aggregated by Investment Category and Length of Time That Individual Securities Have Been in a Continuous Unrealized Loss Position [Abstract] Fair Value, Less than 12 months Unrealized losses, less than 12 months Fair Value Unrealized Losses Fair Value, More than 12 months Unrealized losses, More than 12 months Amorized Cost and Fair Values of Availiable-for-sale Debt Securities by Contractual Maturity [Abstract] Due within one year Due within one year Due after one year through five years Due after one year through five years Due after five years through ten years Due after five years through ten years Due after ten years Due after ten years Total Total Receivables [Abstract] Composition of the Company's Loan Portfolio [Abstract] Financing Receivable Portfolio Segment [Axis] Residential Portfolio Segment [Member] Home Equity Portfolio Segment [Member]