Let $ \vec{a} = 2\hat{i} - \hat{j} + 3\hat{k}, \ \vec{b} = 3\hat{i} - 5\hat{j} + \hat{k} $ and $ \vec{c} $ be a vector such that $ \vec{a} \times \vec{c} = \vec{a} \times \vec{b} = \vec{c} \times \vec{b} $ and $ (\vec{a} + \vec{c}) \cdot (\vec{b} + \vec{c}) = 168 $. Then the maximum value of $|\vec{c}|^2$ is :

<p>$$\begin{aligned} & \overrightarrow{\mathrm{a}}=2 \hat{\mathrm{i}}-\hat{\mathrm{j}}+3 \hat{\mathrm{k}} \\ & \overrightarrow{\mathrm{~b}}=3 \hat{\mathrm{i}}-5 \hat{\mathrm{j}}+\hat{\mathrm{k}} \\ & \overrightarrow{\mathrm{a}} \times \overrightarrow{\mathrm{c}}=\overrightarrow{\mathrm{c}} \times \overrightarrow{\mathrm{b}} \\ & \overrightarrow{\mathrm{a}} \times \overrightarrow{\mathrm{c}}+\overrightarrow{\mathrm{b}} \times \overrightarrow{\mathrm{c}}=0 \\ & (\overrightarrow{\mathrm{a}}+\overrightarrow{\mathrm{b}}) \times \overrightarrow{\mathrm{c}}=0 \\ & \Rightarrow \overrightarrow{\mathrm{c}}=\lambda(\overrightarrow{\mathrm{a}}+\overrightarrow{\mathrm{b}}) \\ & \overrightarrow{\mathrm{c}}=\lambda(5 \hat{\mathrm{i}}-6 \hat{\mathrm{j}}+4 \hat{\mathrm{k}}) \ldots . .(1) \\ & |\overrightarrow{\mathrm{c}}|^2=\lambda^2(25+36+16) \\ & |\overrightarrow{\mathrm{c}}|^2=77 \lambda^2 \\ & (\overrightarrow{\mathrm{a}}+\overrightarrow{\mathrm{c}}) \cdot(\overrightarrow{\mathrm{b}}+\overrightarrow{\mathrm{c}})=168 \\ & \overrightarrow{\mathrm{a}} \cdot \overrightarrow{\mathrm{~b}}+\overrightarrow{\mathrm{a}} \cdot \overrightarrow{\mathrm{c}}+\overrightarrow{\mathrm{c}} \cdot \overrightarrow{\mathrm{~b}}+|\overrightarrow{\mathrm{c}}|^2=168 \\ & 14+\overrightarrow{\mathrm{c}} \cdot(\overrightarrow{\mathrm{a}}+\overrightarrow{\mathrm{b}})+77 \lambda^2=168 \end{aligned}$$</p> <p>using equation (1)</p> <p>$$\begin{aligned} & \lambda|5 \hat{\mathrm{i}}-6 \hat{\mathrm{j}}+4 \hat{\mathrm{k}}|^2+77 \lambda^2=154 \\ & 77 \lambda+77 \lambda^2-154=0 \\ & \lambda^2+\lambda-2=0 \\ & \lambda=-2,1 \end{aligned}$$</p> <p>$\therefore$ Maximum value of $|\overrightarrow{\mathrm{c}}|^2$ occurs when $\lambda=-2$</p> <p>$$\begin{aligned} & |\overrightarrow{\mathrm{c}}|^2=77 \lambda^2 \\ & =77 \times 4 \\ & =308 \end{aligned}$$</p>